I2P_Developers/i2p.www
3
0
Fork 5
Code Issues 27 Pull Requests Actions 1 Packages Projects Releases Wiki Activity

improve html..

Browse Source
This commit is contained in:
dev
2010-07-30 14:26:34 +00:00
parent f8d84e52c0
commit 76a9052cff
1 changed files with 521 additions and 422 deletions
Download Patch File Download Diff File Expand all files Collapse all files

207
www.i2p2/pages/how_networkdatabase.html
View File

@ -4,10 +4,12 @@
<p> <p>
Updated July 2010, current as of router version 0.8 Updated July 2010, current as of router version 0.8
</p>
<h2>Overview</h2> <h2>Overview</h2>
<p>I2P's netDb is a specialized distributed database, containing <p>
I2P's netDb is a specialized distributed database, containing
just two types of data - router contact information (<b>RouterInfos</b>) and destination contact just two types of data - router contact information (<b>RouterInfos</b>) and destination contact
information (<b>LeaseSets</b>). Each piece of data is signed by the appropriate party and verified information (<b>LeaseSets</b>). Each piece of data is signed by the appropriate party and verified
by anyone who uses or stores it. In addition, the data has liveliness information by anyone who uses or stores it. In addition, the data has liveliness information
@ -21,12 +23,15 @@ where a subset of all routers, called "floodfill routers", maintains the distrib
</p> </p>
<h2><a name="routerInfo">RouterInfo</a></h2> <h2 id="routerInfo">RouterInfo</h2>
<p>When an I2P router wants to contact another router, they need to know some <p>
When an I2P router wants to contact another router, they need to know some
key pieces of data - all of which are bundled up and signed by the router into key pieces of data - all of which are bundled up and signed by the router into
a structure called the "RouterInfo", which is distributed under the key derived a structure called the "RouterInfo", which is distributed under the key derived
from the SHA256 of the router's identity. The structure itself contains:</p><ul> from the SHA256 of the router's identity. The structure itself contains:
</p>
<ul>
<li>The router's identity (a 2048bit ElGamal encryption key, a 1024bit DSA signing key, and a certificate)</li> <li>The router's identity (a 2048bit ElGamal encryption key, a 1024bit DSA signing key, and a certificate)</li>
<li>The contact addresses at which it can be reached (e.g. TCP: example.org port 4108)</li> <li>The contact addresses at which it can be reached (e.g. TCP: example.org port 4108)</li>
<li>When this was published</li> <li>When this was published</li>
@ -37,6 +42,7 @@ from the SHA256 of the router's identity. The structure itself contains:</p><ul
<p> <p>
The following text options, while not strictly required, are expected The following text options, while not strictly required, are expected
to be present: to be present:
</p>
<ul> <ul>
<li><b>caps</b> <li><b>caps</b>
(Capabilities flags - used to indicate floodfill participation, approximate bandwidth, and perceived reachability) (Capabilities flags - used to indicate floodfill participation, approximate bandwidth, and perceived reachability)
@ -51,16 +57,18 @@ to be present:
and only sent tunnel requests to peers whose was more than 60m) and only sent tunnel requests to peers whose was more than 60m)
</ul> </ul>
<p>
These values are used by other routers for basic decisions. These values are used by other routers for basic decisions.
Should we connect to this router? Should we attempt to route a tunnel through this router? Should we connect to this router? Should we attempt to route a tunnel through this router?
The bandwidth capability flag, in particular, is used only to determine whether The bandwidth capability flag, in particular, is used only to determine whether
the router meets a minimum threshold for routing tunnels. the router meets a minimum threshold for routing tunnels.
Above the minimum threshold, the advertised bandwidth is not used or trusted anywhere Above the minimum threshold, the advertised bandwidth is not used or trusted anywhere
in the router, except for display in the user interface and for debugging and network analysis. in the router, except for display in the user interface and for debugging and network analysis.
</p>
<p>
<p>Additional text options include Additional text options include
a small number of statistics about the router's health, which are aggregated by a small number of statistics about the router's health, which are aggregated by
sites such as <a href="http://stats.i2p/">stats.i2p</a> sites such as <a href="http://stats.i2p/">stats.i2p</a>
for network performance analysis and debugging. for network performance analysis and debugging.
@ -68,31 +76,35 @@ These statistics were chosen to provide data crucial to the developers,
such as tunnel build success rates, while balancing the need for such data such as tunnel build success rates, while balancing the need for such data
with the side-effects that could result from revealing this data. with the side-effects that could result from revealing this data.
Current statistics are limited to: Current statistics are limited to:
</p>
<ul> <ul>
<li>1 hour average bandwidth (average of outbound and inbound bandwidth) <li>1 hour average bandwidth (average of outbound and inbound bandwidth)
<li>Client and exporatory tunnel build success, reject, and timeout rates <li>Client and exporatory tunnel build success, reject, and timeout rates
<li>1 hour average number of participating tunnels <li>1 hour average number of participating tunnels
</ul> </ul>
<p>
The data The data
published can be seen in the router's user interface, published can be seen in the router's user interface,
but is not used or trusted within the router. but is not used or trusted within the router.
As the network has matured, we have gradually removed most of the published As the network has matured, we have gradually removed most of the published
statistics to improve anonymity, and we plan to remove more in future releases. statistics to improve anonymity, and we plan to remove more in future releases.
</p> </p>
<p> <p>
<a href="common_structures_spec.html#struct_RouterInfo">RouterInfo specification</a> <a href="common_structures_spec.html#struct_RouterInfo">RouterInfo specification</a>
</p>
<p> <p>
<a href="http://docs.i2p2.de/core/net/i2p/data/RouterInfo.html">RouterInfo Javadoc</a> <a href="http://docs.i2p2.de/core/net/i2p/data/RouterInfo.html">RouterInfo Javadoc</a>
</p>
<h3>RouterInfo Expiration</a></h3> <h3>RouterInfo Expiration</h3>
<p>
RouterInfos have no set expiration time. RouterInfos have no set expiration time.
Each router is free to maintain its own local policy to trade off the frequency of RouterInfo lookups Each router is free to maintain its own local policy to trade off the frequency of RouterInfo lookups
with memory or disk usage. with memory or disk usage.
In the current implementation, there are the following general policies: In the current implementation, there are the following general policies:
</p>
<ul> <ul>
<li>There is no expiration during the first hour of uptime, as the perstistent stored data may be old. <li>There is no expiration during the first hour of uptime, as the perstistent stored data may be old.
<li>As the number of local RouterInfos grows, the expiration time shrinks, in an attempt to maintain <li>As the number of local RouterInfos grows, the expiration time shrinks, in an attempt to maintain
@ -103,21 +115,26 @@ In the current implementation, there are the following general policies:
be frequently republished to them be frequently republished to them
</ul> </ul>
<h3>RouterInfo Persistent Storage</a></h3> <h3>RouterInfo Persistent Storage</h3>
<p>
RouterInfos are periodically written to disk so that they are available after a restart. RouterInfos are periodically written to disk so that they are available after a restart.
</p>
<h2><a name="leaseSet">LeaseSet</a></h2> <h2 id="leaseSet"LeaseSet</h2>
<p>The second piece of data distributed in the netDb is a "LeaseSet" - documenting <p>
The second piece of data distributed in the netDb is a "LeaseSet" - documenting
a group of tunnel entry points (leases) for a particular client destination. a group of tunnel entry points (leases) for a particular client destination.
Each of these leases specify the tunnel's gateway router (with the hash of its Each of these leases specify the tunnel's gateway router (with the hash of its
identity), the tunnel ID on that router to send messages (a 4 byte number), and identity), the tunnel ID on that router to send messages (a 4 byte number), and
when that tunnel will expire. The LeaseSet itself is stored in the netDb under when that tunnel will expire. The LeaseSet itself is stored in the netDb under
the key derived from the SHA256 of the destination.</p> the key derived from the SHA256 of the destination.
</p>
<p>In addition to these leases, the LeaseSet includes the destination <p>
In addition to these leases, the LeaseSet includes the destination
itself (namely, the destination's 2048bit ElGamal encryption key, 1024bit DSA itself (namely, the destination's 2048bit ElGamal encryption key, 1024bit DSA
signing key, and certificate) and an additional signing and signing key, and certificate) and an additional signing and
encryption public keys. The additional encryption public key is used for end-to-end encryption of encryption public keys. The additional encryption public key is used for end-to-end encryption of
@ -127,54 +144,58 @@ The additional signing publc key was intended for LeaseSet revocation but is cur
<p> <p>
<a href="common_structures_spec.html#struct_Lease">Lease specification</a> <a href="common_structures_spec.html#struct_Lease">Lease specification</a>
<br> <br />
<a href="common_structures_spec.html#struct_LeaseSet">LeaseSet specification</a> <a href="common_structures_spec.html#struct_LeaseSet">LeaseSet specification</a>
</p>
<p> <p>
<a href="http://docs.i2p2.de/core/net/i2p/data/Lease.html">Lease Javadoc</a> <a href="http://docs.i2p2.de/core/net/i2p/data/Lease.html">Lease Javadoc</a>
<br> <br />
<a href="http://docs.i2p2.de/core/net/i2p/data/LeaseSet.html">LeaseSet Javadoc</a> <a href="http://docs.i2p2.de/core/net/i2p/data/LeaseSet.html">LeaseSet Javadoc</a>
</p>
<h3><a name="unpublished">Unpublished LeaseSets</a></h3> <h3 id="unpublished">Unpublished LeaseSets</h3>
<p>
A LeaseSet for a destination used only for outgoing connections is <i>unpublished</i>. A LeaseSet for a destination used only for outgoing connections is <i>unpublished</i>.
It is never sent for publication to a floodfill router. It is never sent for publication to a floodfill router.
"Client" tunnels, such as those for web browsing and IRC clients, are unpublished. "Client" tunnels, such as those for web browsing and IRC clients, are unpublished.
</p>
<h3><a name="revoked">Revoked LeaseSets</a></h3> <h3 id="revoked">Revoked LeaseSets</h3>
<p>
A LeaseSet may be <i>revoked</i> by publishing a new LeaseSet with zero leases. A LeaseSet may be <i>revoked</i> by publishing a new LeaseSet with zero leases.
Revocations must be signed by the additional signing key in the leaseset. Revocations must be signed by the additional signing key in the leaseset.
Revocations are not fully implemented, and it is unclear if they have any practical use. Revocations are not fully implemented, and it is unclear if they have any practical use.
This is the only planned use for the that signing key, so it is currently unused. This is the only planned use for the that signing key, so it is currently unused.
</p>
<h3 id="encrypted">Encrypted LeaseSets</h3>
<h3><a name="encrypted">Encrypted LeaseSets</a></h3> <p>
In an <i>encrypted</i> LeaseSet, all Leases are encrypted with a separate DSA key. In an <i>encrypted</i> LeaseSet, all Leases are encrypted with a separate DSA key.
The leases may only be decoded, and thus the destination may only be contacted, The leases may only be decoded, and thus the destination may only be contacted,
by those with the key. by those with the key.
There is no flag or other direct indication that the LeaseSet is encrypted. There is no flag or other direct indication that the LeaseSet is encrypted.
Encrypted LeaseSets are not widely used, and it is a topic for future work to Encrypted LeaseSets are not widely used, and it is a topic for future work to
research whether the user interface and implementation of encrypted leasesets could be improved. research whether the user interface and implementation of encrypted leasesets could be improved.
</p>
<h3>LeaseSet Expiration</h3>
<h3>LeaseSet Expiration</a></h3> <p>
All Leases (tunnels) are valid for 10 minutes; therefore, a LeaseSet expires All Leases (tunnels) are valid for 10 minutes; therefore, a LeaseSet expires
10 minutes after the earliest creation time of all its Leases. 10 minutes after the earliest creation time of all its Leases.
</p>
<h3>LeaseSet Persistent Storage</a></h3> <h3>LeaseSet Persistent Storage</h3>
<p>
There is no persistent storage of LeaseSet data since they expire so quickly. There is no persistent storage of LeaseSet data since they expire so quickly.
</p>
<h2 id="bootstrap">Bootstrapping</h2>
<h2><a name="bootstrap">Bootstrapping</a></h2> <p>
The netDb is decentralized, however you do need at
<p>The netDb is decentralized, however you do need at
least one reference to a peer so that the integration process least one reference to a peer so that the integration process
ties you in. This is accomplished by "reseeding" your router with the RouterInfo ties you in. This is accomplished by "reseeding" your router with the RouterInfo
of an active peer - specifically, by retrieving their <code>routerInfo-$hash.dat</code> of an active peer - specifically, by retrieving their <code>routerInfo-$hash.dat</code>
@ -187,7 +208,8 @@ When the router starts up for the first time, it automatically fetches from
one of these URLs, selected at random. one of these URLs, selected at random.
</p> </p>
<h2><a name="floodfill">Floodfill</a></h2> <h2 id="floodfill">Floodfill</h2>
<p>
The floodfill netDb is simple distributed storage mechanism. The floodfill netDb is simple distributed storage mechanism.
The storage algorithm is simple- send the data to the closest peer that has advertised itself The storage algorithm is simple- send the data to the closest peer that has advertised itself
as a floodfill router. Then wait 10 seconds, pick another floodfill router and ask them as a floodfill router. Then wait 10 seconds, pick another floodfill router and ask them
@ -196,21 +218,26 @@ verification peer doesn't reply, or they don't have the entry, the sender
repeats the process. When the peer in the floodfill netDb receives a netDb repeats the process. When the peer in the floodfill netDb receives a netDb
store from a peer not in the floodfill netDb, they send it to all of the peers store from a peer not in the floodfill netDb, they send it to all of the peers
in the floodfill netDb. in the floodfill netDb.
</p><p> </p>
<p>
Determining who is part of the floodfill netDb is trivial - it is exposed in each Determining who is part of the floodfill netDb is trivial - it is exposed in each
router's published routerInfo as a capability. router's published routerInfo as a capability.
</p><p> </p>
<p>
Floodfills have no central authority and do not form a "consensus" - Floodfills have no central authority and do not form a "consensus" -
they only implement a simple DHT overlay. they only implement a simple DHT overlay.
</p>
<h3><a name="opt-in">Floodfill Router Opt-in</a></h3>
<h3 id="opt-in">Floodfill Router Opt-in</h3>
<p> <p>
Unlike Tor, where the directory servers are hardcoded and trusted, Unlike Tor, where the directory servers are hardcoded and trusted,
and operated by known entities, and operated by known entities,
the members of the I2P floodfill peer set need not be trusted, and the members of the I2P floodfill peer set need not be trusted, and
change over time. change over time.
</p>
<p> <p>
To increase reliability of the netDb, and minimize the impact To increase reliability of the netDb, and minimize the impact
of netDb traffic on a router, floodfill is automatically enabled of netDb traffic on a router, floodfill is automatically enabled
@ -219,14 +246,18 @@ Routers with high bandwidth limits (which must be manually configured,
as the default is much lower) are presumed to be on lower-latency as the default is much lower) are presumed to be on lower-latency
connections, and are more likely to be available 24/7. connections, and are more likely to be available 24/7.
The current minimum share bandwidth for a floodfill router is 128 KBytes/sec. The current minimum share bandwidth for a floodfill router is 128 KBytes/sec.
</p>
<p> <p>
In addition, a router must pass several additional tests for health In addition, a router must pass several additional tests for health
(outbound message queue time, job lag, etc.) before floodfill operation is (outbound message queue time, job lag, etc.) before floodfill operation is
automatically enabled. automatically enabled.
</p>
<p> <p>
With the current rules for automatic opt-in, approximately 6% of With the current rules for automatic opt-in, approximately 6% of
the routers in the network are floodfill routers. the routers in the network are floodfill routers.
</p>
<p>
While some peers are manually configured to be floodfill, While some peers are manually configured to be floodfill,
others are simply high-bandwidth routers who automatically volunteer others are simply high-bandwidth routers who automatically volunteer
when the number of floodfill peers drops below a threshold. when the number of floodfill peers drops below a threshold.
@ -234,44 +265,52 @@ This prevents any long-term network damage from losing most or all
floodfills to an attack. floodfills to an attack.
In turn, these peers will un-floodfill themselves when there are In turn, these peers will un-floodfill themselves when there are
too many floodfills outstanding. too many floodfills outstanding.
</p>
<h3>Floodfill Router Roles</h3> <h3>Floodfill Router Roles</h3>
<p>
A floodfill router's only services that are in addition to those of non-floodfill routers A floodfill router's only services that are in addition to those of non-floodfill routers
are in accepting netDb stores and responding to netDb queries. are in accepting netDb stores and responding to netDb queries.
Since they are generally high-bandwidth, they are more likely to participate in a high number of tunnels Since they are generally high-bandwidth, they are more likely to participate in a high number of tunnels
(i.e. be a "relay" for others), but this not directly related to their distributed database services. (i.e. be a "relay" for others), but this not directly related to their distributed database services.
</p>
<h2><a name="kad">Kademlia Closeness Metric</a></h2> <h2 id="kad">Kademlia Closeness Metric</h2>
<p>
The netDb uses a simple Kademlia-style XOR metric to determine closeness. The netDb uses a simple Kademlia-style XOR metric to determine closeness.
The SHA256 Hash of the key being looked up or stored is XORed with The SHA256 Hash of the key being looked up or stored is XORed with
the hash of the router in question to determine closeness the hash of the router in question to determine closeness
(there is an additional daily keyspace rotation to increase the costs of sybil attacks, (there is an additional daily keyspace rotation to increase the costs of sybil attacks,
as <a href="#sybil-partial">explained below). as <a href="#sybil-partial">explained below).
</p>
<h2><a name="delivery">Storage, Verification, and Lookup Mechanics</a></h2> <h2 id="delivery">Storage, Verification, and Lookup Mechanics</h2>
<h3>RouterInfo Storage to Peers</h3> <h3>RouterInfo Storage to Peers</h3>
<p>
<a href="i2np.html">I2NP</a> DatabaseStoreMessages containing the local RouterInfo are exchanged with peers <a href="i2np.html">I2NP</a> DatabaseStoreMessages containing the local RouterInfo are exchanged with peers
as a part of the initialization of a as a part of the initialization of a
<a href="ntcp.html">NTCP</a> <a href="ntcp.html">NTCP</a>
or or
<a href="udp.html">SSU</a> <a href="udp.html">SSU</a>
transport connection. transport connection.
</p>
<h3>LeaseSet Storage to Peers</h3> <h3>LeaseSet Storage to Peers</h3>
<p>
<a href="i2np.html">I2NP</a> DatabaseStoreMessages containing the local LeaseSet are periodically exchanged with peers <a href="i2np.html">I2NP</a> DatabaseStoreMessages containing the local LeaseSet are periodically exchanged with peers
by bundling them in a garlic message along with normal traffic from the related Destination. by bundling them in a garlic message along with normal traffic from the related Destination.
This allows an initial response, and later responses, to be sent to an appropriate Lease, This allows an initial response, and later responses, to be sent to an appropriate Lease,
without requiring any LeaseSet lookups, or requiring the communicating Destinations to have published LeaseSets at all. without requiring any LeaseSet lookups, or requiring the communicating Destinations to have published LeaseSets at all.
</p>
<h3>RouterInfo Storage to Floodfills</h3> <h3>RouterInfo Storage to Floodfills</h3>
<p>
A router publishes its own RouterInfo by directly connecting to a floodfill router A router publishes its own RouterInfo by directly connecting to a floodfill router
and sending it a <a href="i2np.html">I2NP</a> DatabaseStoreMessage and sending it a <a href="i2np.html">I2NP</a> DatabaseStoreMessage
with a nonzero Reply Token. The message is not end-to-end garlic encrypted, with a nonzero Reply Token. The message is not end-to-end garlic encrypted,
@ -280,16 +319,18 @@ as this is a direct connection, so there are no intervening routers
The floodfill router replies with a The floodfill router replies with a
<a href="i2np.html">I2NP</a> DeliveryStatusMessage, <a href="i2np.html">I2NP</a> DeliveryStatusMessage,
with the Message ID set to the value of the Reply Token. with the Message ID set to the value of the Reply Token.
</p>
<h3>LeaseSet Storage to Floodfills</h3> <h3>LeaseSet Storage to Floodfills</h3>
<p>
Storage of LeaseSets is much more sensitive than for RouterInfos, as a router Storage of LeaseSets is much more sensitive than for RouterInfos, as a router
must take care that the LeaseSet cannot be associated with the router. must take care that the LeaseSet cannot be associated with the router.
<p> </p>
<p>
A router publishes its own RouterInfo by directly connecting to a floodfill router A router publishes its own RouterInfo by directly connecting to a floodfill router
and sending it a <a href="i2np.html">I2NP</a> DatabaseStoreMessage and sending it a <a href="i2np.html">I2NP</a> DatabaseStoreMessage
with a nonzero Reply Token. The message is not end-to-end garlic encrypted, with a nonzero Reply Token. The message is not end-to-end garlic encrypted,
@ -298,6 +339,7 @@ as this is a direct connection, so there are no intervening routers
The floodfill router replies with a The floodfill router replies with a
<a href="i2np.html">I2NP</a> DeliveryStatusMessage, <a href="i2np.html">I2NP</a> DeliveryStatusMessage,
with the Message ID set to the value of the Reply Token. with the Message ID set to the value of the Reply Token.
</p>
@ -306,11 +348,13 @@ with the Message ID set to the value of the Reply Token.
<h3>Flooding</h3> <h3>Flooding</h3>
<p>
After a floodfill router receives a DatabaseStoreMessage containing a After a floodfill router receives a DatabaseStoreMessage containing a
valid RouterInfo or LeaseSet which is newer than that previously stored in its valid RouterInfo or LeaseSet which is newer than that previously stored in its
local NetDb, it "floods" it. local NetDb, it "floods" it.
To flood a NetDb entry, it looks up the 7 floodfill routers closest to the key To flood a NetDb entry, it looks up the 7 floodfill routers closest to the key
of the NetDb entry. (The key is the SHA256 Hash of the RouterIdentity or Destination) of the NetDb entry. (The key is the SHA256 Hash of the RouterIdentity or Destination)
</p>
<p> <p>
It then directly connects to each of the 7 peers It then directly connects to each of the 7 peers
and sends it a <a href="i2np.html">I2NP</a> DatabaseStoreMessage and sends it a <a href="i2np.html">I2NP</a> DatabaseStoreMessage
@ -318,63 +362,77 @@ with a zero Reply Token. The message is not end-to-end garlic encrypted,
as this is a direct connection, so there are no intervening routers as this is a direct connection, so there are no intervening routers
(and no need to hide this data anyway). (and no need to hide this data anyway).
The other routers do not reply or re-flood, as the Reply Token is zero. The other routers do not reply or re-flood, as the Reply Token is zero.
</p>
<h3><a name="lookup">RouterInfo and LeaseSet Lookup</a></h3> <h3 id="lookup">RouterInfo and LeaseSet Lookup</h3>
<p>
The <a href="i2np.html">I2NP</a> DatabaseLookupMessage is used to request a netdb entry from a floodfill router. The <a href="i2np.html">I2NP</a> DatabaseLookupMessage is used to request a netdb entry from a floodfill router.
Lookups are sent out one of the router's outbound exploratory tunnels. Lookups are sent out one of the router's outbound exploratory tunnels.
The replies are specified to return via one of the router's inbound exploratory tunnels. The replies are specified to return via one of the router's inbound exploratory tunnels.
</p>
<p> <p>
Lookups are generally sent to the two "good" floodfill routers closest to the requested key, in parallel. Lookups are generally sent to the two "good" floodfill routers closest to the requested key, in parallel.
</p>
<p> <p>
If the key is found locally by the floodfill router, it responds with a If the key is found locally by the floodfill router, it responds with a
<a href="i2np.html">I2NP</a> DatabaseStoreMessage. <a href="i2np.html">I2NP</a> DatabaseStoreMessage.
If the key is not found locally by the floodfill router, it responds with a If the key is not found locally by the floodfill router, it responds with a
<a href="i2np.html">I2NP</a> DatabaseSearchReplyMessage <a href="i2np.html">I2NP</a> DatabaseSearchReplyMessage
containing a list of other floodfill routers close to the key. containing a list of other floodfill routers close to the key.
</p>
<p> <p>
Lookups are not encrypted and thus are vulnerable to snooping by the outbound endpoint Lookups are not encrypted and thus are vulnerable to snooping by the outbound endpoint
(OBEP) of the client tunnel. (OBEP) of the client tunnel.
</p>
<p> <p>
As the requesting router does not reveal itself, there is no recipient public key for the floodfill router to As the requesting router does not reveal itself, there is no recipient public key for the floodfill router to
encrypt the reply with. Therefore, the reply is exposed to the inbound gateway (IBGW) encrypt the reply with. Therefore, the reply is exposed to the inbound gateway (IBGW)
of the inbound exploratory tunnel. of the inbound exploratory tunnel.
An appropriate method of encrypting the reply is a topic for future work. An appropriate method of encrypting the reply is a topic for future work.
</p>
<p> <p>
(Reference: (Reference:
<a href="http://www-users.cs.umn.edu/~hopper/hashing_it_out.pdf">Hashing it out in Public</a> Section 2.3 for terms below in italics) <a href="http://www-users.cs.umn.edu/~hopper/hashing_it_out.pdf">Hashing it out in Public</a> Section 2.3 for terms below in italics)
</p>
<p> <p>
Due to the relatively small size of the network, the flooding redundancy of 8x, and a lookup redundancy of 2x, Due to the relatively small size of the network, the flooding redundancy of 8x, and a lookup redundancy of 2x,
lookups are currently O(1) rather than O(log n) -- lookups are currently O(1) rather than O(log n) --
a router is highly likely to know a floodfill router close enough to the key to get the answer on the first try. a router is highly likely to know a floodfill router close enough to the key to get the answer on the first try.
Neither <i>recursive</i> nor <i>iterative</i> routing for lookups is implemented. Neither <i>recursive</i> nor <i>iterative</i> routing for lookups is implemented.
</p>
<p> <p>
<i>Node IDs</i> are <i>verifiable</i> in that we use the router hash directly as both the node ID and the Kademlia key. <i>Node IDs</i> are <i>verifiable</i> in that we use the router hash directly as both the node ID and the Kademlia key.
Given the current size of the network, a router has Given the current size of the network, a router has
<i>detailed knowledge of the neighborhood of the destination ID space</i>. <i>detailed knowledge of the neighborhood of the destination ID space</i>.
</p>
<p> <p>
Queries are sent through<i>multiple routes simultaneously</i> Queries are sent through<i>multiple routes simultaneously</i>
to <i>reduce the chance of query failure</i>. to <i>reduce the chance of query failure</i>.
</p>
<p> <p>
After network growth of 5x - 10x, there will be a significant chance of lookup failure due After network growth of 5x - 10x, there will be a significant chance of lookup failure due
to the O(1) lookup strategy, and implementation of an <i>iterative</i> lookup strategy will be required. to the O(1) lookup strategy, and implementation of an <i>iterative</i> lookup strategy will be required.
See <a href="#future">below</a> for more information. See <a href="#future">below</a> for more information.
</p>
<h3>RouterInfo Storage Verification</h3> <h3>RouterInfo Storage Verification</h3>
<p>
To verify a storage was successful, a router simply waits about 10 seconds, To verify a storage was successful, a router simply waits about 10 seconds,
then sends a lookup to another floodfill router close to the key then sends a lookup to another floodfill router close to the key
(but not the one the store was sent to). (but not the one the store was sent to).
Lookups sent out one of the router's outbound exploratory tunnels. Lookups sent out one of the router's outbound exploratory tunnels.
Lookups are end-to-end garlic encrypted and to prevent snooping by the O tunnels. Lookups are end-to-end garlic encrypted and to prevent snooping by the O tunnels.
</p>
<h3>LeaseSet Storage Verification</h3> <h3>LeaseSet Storage Verification</h3>
<p>
To verify a storage was successful, a router simply waits about 10 seconds, To verify a storage was successful, a router simply waits about 10 seconds,
then sends a lookup to another floodfill router close to the key then sends a lookup to another floodfill router close to the key
(but not the one the store was sent to). (but not the one the store was sent to).
@ -382,16 +440,19 @@ Lookups sent out one of the outbound client tunnels for the destination of the L
To prevent snooping by the OBEP of the outbound tunnel, To prevent snooping by the OBEP of the outbound tunnel,
lookups are end-to-end garlic encrypted. lookups are end-to-end garlic encrypted.
The replies are specified to return via one of the client's inbound tunnels. The replies are specified to return via one of the client's inbound tunnels.
</p>
<p> <p>
As for regular lookups, the reply is unencrypted, As for regular lookups, the reply is unencrypted,
thus exposing the reply to the inbound gateway (IBGW) of the reply tunnel, and thus exposing the reply to the inbound gateway (IBGW) of the reply tunnel, and
an appropriate method of encrypting the reply is a topic for future work. an appropriate method of encrypting the reply is a topic for future work.
As the IBGW for the reply is one of the gateways published in the LeaseSet, As the IBGW for the reply is one of the gateways published in the LeaseSet,
the exposure is minimal. the exposure is minimal.
</p>
<h3>Exploration</h3> <h3>Exploration</h3>
<p>
<i>Exploration</i> is a special form of netdb lookup, where a router attempts to learn about <i>Exploration</i> is a special form of netdb lookup, where a router attempts to learn about
new routers. new routers.
It does this by sending a floodfill router a <a href="i2np.html">I2NP</a> DatabaseLookupMessage, looking for a random key. It does this by sending a floodfill router a <a href="i2np.html">I2NP</a> DatabaseLookupMessage, looking for a random key.
@ -402,11 +463,13 @@ and it would be impractical to add ALL floodfill routers to the "don't include"
For an exploration query, the requesting router adds a router hash of all zeros to the For an exploration query, the requesting router adds a router hash of all zeros to the
"don't include" field of the DatabaseLookupMessage. "don't include" field of the DatabaseLookupMessage.
The floodfill will then respond only with non-floodfill routers close to the requested key. The floodfill will then respond only with non-floodfill routers close to the requested key.
</p>
<h2><a name="multihome">MultiHoming</a></h2> <h2 id="multihome">MultiHoming</h2>
<p>Destinations may be hosted on multiple routers simultaneously, by using the same <p>
Destinations may be hosted on multiple routers simultaneously, by using the same
private and public keys (traditionally named eepPriv.dat files). private and public keys (traditionally named eepPriv.dat files).
As both instances will periodically publish their signed LeaseSets to the floodfill peers, As both instances will periodically publish their signed LeaseSets to the floodfill peers,
the most recently published LeaseSet will be returned to a peer requesting a database lookup. the most recently published LeaseSet will be returned to a peer requesting a database lookup.
@ -415,33 +478,44 @@ the outage will be 10 minutes at most, and generally much less than that.
The multihoming function has been verified and is in use by several services on the network. The multihoming function has been verified and is in use by several services on the network.
</p> </p>
<h2><a name="threat">Threat Analysis</a></h2> <h2 id="threat">Threat Analysis</h2>
<p>
Also discussed on <a href="how_threatmodel.html#floodfill">the threat model page</a>. Also discussed on <a href="how_threatmodel.html#floodfill">the threat model page</a>.
</p>
<p> <p>
A hostile user may attempt to harm the network by A hostile user may attempt to harm the network by
creating one or more floodfill routers and crafting them to offer creating one or more floodfill routers and crafting them to offer
bad, slow, or no reponses. bad, slow, or no reponses.
Some scenarios are discussed below. Some scenarios are discussed below.
</p>
<h3>General Mitigation Through Growth</h3> <h3>General Mitigation Through Growth</h3>
<p>
There are currently almost 100 floodfill routers in the network. There are currently almost 100 floodfill routers in the network.
Most of the following attacks will become more difficult, or have less impact, Most of the following attacks will become more difficult, or have less impact,
as the network size and number of floodfill routers increase. as the network size and number of floodfill routers increase.
</p>
<h3>General Mitigation Through Redundancy</h3> <h3>General Mitigation Through Redundancy</h3>
<p>
Via flooding, all netdb entries are stored on the 8 floodfill routers closest to the key. Via flooding, all netdb entries are stored on the 8 floodfill routers closest to the key.
</p>
<h3>Forgeries</h3> <h3>Forgeries</h3>
<p>
All netdb entries are signed by their creators, so no router may forge a All netdb entries are signed by their creators, so no router may forge a
RouterInfo or LeaseSet. RouterInfo or LeaseSet.
</p>
<h3>Slow or Unresponsive</h3> <h3>Slow or Unresponsive</h3>
<p>
Each router maintains an expanded set of statistics in the Each router maintains an expanded set of statistics in the
<a href="how_peerselection.html">peer profile</a> for each floodfill router, <a href="how_peerselection.html">peer profile</a> for each floodfill router,
covering various quality metrics for that peer. covering various quality metrics for that peer.
The set includes: The set includes:
</p>
<ul> <ul>
<li>Average response time <li>Average response time
<li>Percentage of queries answered with the data requested <li>Percentage of queries answered with the data requested
@ -458,17 +532,22 @@ The methods, and thresholds, used to determine "goodness" are relatively new, an
are subject to further analysis and improvement. are subject to further analysis and improvement.
While a completely unresponsive router will quickly be identified and avoided, While a completely unresponsive router will quickly be identified and avoided,
routers that are only sometimes malicious may be much harder to deal with. routers that are only sometimes malicious may be much harder to deal with.
</p>
<h3>Sybil Attack (Full Keyspace)</h3> <h3>Sybil Attack (Full Keyspace)</h3>
<p>
An attacker may mount a An attacker may mount a
<a href="http://citeseer.ist.psu.edu/douceur02sybil.html">Sybil attack</a> <a href="http://citeseer.ist.psu.edu/douceur02sybil.html">Sybil attack</a>
by creating a large number of floodfill routers spread throughout the keyspace. by creating a large number of floodfill routers spread throughout the keyspace.
</p>
<p>
(In a related example, a researcher recently created a (In a related example, a researcher recently created a
<a href="http://blog.torproject.org/blog/june-2010-progress-report">large number of Tor relays</a>.) <a href="http://blog.torproject.org/blog/june-2010-progress-report">large number of Tor relays</a>.)
If successful, this could be an effective DOS attack on the entire network. If successful, this could be an effective DOS attack on the entire network.
</p>
<p> <p>
If the floodfills are not sufficiently misbehaving to be marked as "bad" using the peer profile If the floodfills are not sufficiently misbehaving to be marked as "bad" using the peer profile
@ -476,6 +555,7 @@ metrics described above, this is a difficult scenario to handle.
Tor's response can be much more nimble in the relay case, as the suspicious relays Tor's response can be much more nimble in the relay case, as the suspicious relays
can be manually removed from the consensus. can be manually removed from the consensus.
Some possible reponses in the I2P case, none of them satisfactory: Some possible reponses in the I2P case, none of them satisfactory:
</p>
<ul> <ul>
<li>Compile a list of bad router hashes or IPs, and announce the list through various means <li>Compile a list of bad router hashes or IPs, and announce the list through various means
(console news, website, forum, etc.); users would have to manually download the list and (console news, website, forum, etc.); users would have to manually download the list and
@ -494,10 +574,12 @@ Some possible reponses in the I2P case, none of them satisfactory:
<p> <p>
This attack becomes more difficult as the network size grows. This attack becomes more difficult as the network size grows.
</p>
<h3><a name="sybil-partial">Sybil Attack (Partial Keyspace)</a></h3> <h3 id="sybil-partial">Sybil Attack (Partial Keyspace)</h3>
<p>
An attacker may mount a An attacker may mount a
<a href="http://citeseer.ist.psu.edu/douceur02sybil.html">Sybil attack</a> <a href="http://citeseer.ist.psu.edu/douceur02sybil.html">Sybil attack</a>
by creating a small number (8-15) of floodfill routers clustered closely in the keyspace, by creating a small number (8-15) of floodfill routers clustered closely in the keyspace,
@ -505,12 +587,14 @@ and distribute the RouterInfos for these routers widely.
Then, all lookups and stores for a key in that keyspace would be directed Then, all lookups and stores for a key in that keyspace would be directed
to one of the attacker's routers. to one of the attacker's routers.
If successful, this could be an effective DOS attack on a particular eepsite, for example. If successful, this could be an effective DOS attack on a particular eepsite, for example.
</p>
<p> <p>
As the keyspace is indexed by the cryptographic (SHA256) Hash of the key, As the keyspace is indexed by the cryptographic (SHA256) Hash of the key,
an attacker must use a brute-force method to repeatedly generate router hashes an attacker must use a brute-force method to repeatedly generate router hashes
until he has enough that are sufficiently close to the key. until he has enough that are sufficiently close to the key.
The amount of computational power required for this, which is dependent on network The amount of computational power required for this, which is dependent on network
size, is unknown. size, is unknown.
</p>
<p> <p>
As a partial defense against this attack, As a partial defense against this attack,
@ -522,9 +606,10 @@ In other words, the entire netdb keyspace "rotates" every day at UTC midnight.
Any partial-keyspace attack would have to be regenerated every day, as Any partial-keyspace attack would have to be regenerated every day, as
after the rotation, the attacking routers would no longer be close after the rotation, the attacking routers would no longer be close
to the target key, or to each other. to the target key, or to each other.
</p>
<p> <p>
This attack becomes more difficult as the network size grows. This attack becomes more difficult as the network size grows.
</p>
<p> <p>
One consequence of daily keyspace rotation is that the distributed network database One consequence of daily keyspace rotation is that the distributed network database
@ -533,16 +618,19 @@ lookups will fail because the new "closest" router has not received a store yet.
The extent of the issue, and methods for mitigation The extent of the issue, and methods for mitigation
(for example netdb "handoffs" at midnight) (for example netdb "handoffs" at midnight)
are a topic for further study. are a topic for further study.
</p>
<h3>Bootstrap Attacks</h3> <h3>Bootstrap Attacks</h3>
<p>
An attacker could attempt to boot new routers into an isolated An attacker could attempt to boot new routers into an isolated
or majority-controlled network by taking over a reseed website, or majority-controlled network by taking over a reseed website,
or tricking the developers into adding his reseed website or tricking the developers into adding his reseed website
to the hardcoded list in the router. to the hardcoded list in the router.
</p>
<p> <p>
Several defenses are possible, and most of these are planned: Several defenses are possible, and most of these are planned:
</p>
<ul> <ul>
<li>Switching from http to https for reseeding, with SSL certificate verification <li>Switching from http to https for reseeding, with SSL certificate verification
<li>Changing the reseed task to fetch a subset of RouterInfos from <li>Changing the reseed task to fetch a subset of RouterInfos from
@ -554,17 +642,21 @@ Several defenses are possible, and most of these are planned:
</ul> </ul>
<h3>Query Capture</h3> <h3>Query Capture</h3>
<p>
See also <a href="#lookup">lookup</a> See also <a href="#lookup">lookup</a>
(Reference: (Reference:
<a href="http://www-users.cs.umn.edu/~hopper/hashing_it_out.pdf">Hashing it out in Public</a> Section 2.3 for terms below in italics) <a href="http://www-users.cs.umn.edu/~hopper/hashing_it_out.pdf">Hashing it out in Public</a> Section 2.3 for terms below in italics)
</p>
<p> <p>
Similar to a bootstrap attack, an attacker using a floodfill router could attempt to "steer" Similar to a bootstrap attack, an attacker using a floodfill router could attempt to "steer"
peers to a subset of routers controlled by him by returning their references. peers to a subset of routers controlled by him by returning their references.
</p>
<p> <p>
This is unlikely to work via exploration, because exploration is a low-frequency task. This is unlikely to work via exploration, because exploration is a low-frequency task.
Routers acquire a majority of their peer references through normal tunnel building activity. Routers acquire a majority of their peer references through normal tunnel building activity.
Exploration results are generally limited to a few router hashes, Exploration results are generally limited to a few router hashes,
and each exploration query is directed to a random floodfill router. and each exploration query is directed to a random floodfill router.
</p>
<p> <p>
For floodfill router references returned in a For floodfill router references returned in a
<a href="i2np.html">I2NP</a> DatabaseSearchReplyMessage <a href="i2np.html">I2NP</a> DatabaseSearchReplyMessage
@ -576,26 +668,33 @@ In other words, the Kademlia lookup is not iterative.
This means the query capture attack described in This means the query capture attack described in
<a href="http://www-users.cs.umn.edu/~hopper/hashing_it_out.pdf">Hashing it out in Public</a> <a href="http://www-users.cs.umn.edu/~hopper/hashing_it_out.pdf">Hashing it out in Public</a>
much less likely, until <a href="future">iterative lookups are implemented</a>. much less likely, until <a href="future">iterative lookups are implemented</a>.
</p>
<h3>DHT-Based Relay Selection</h3> <h3>DHT-Based Relay Selection</h3>
<p>
(Reference: (Reference:
<a href="http://www-users.cs.umn.edu/~hopper/hashing_it_out.pdf">Hashing it out in Public</a> Section 3) <a href="http://www-users.cs.umn.edu/~hopper/hashing_it_out.pdf">Hashing it out in Public</a> Section 3)
</p>
<p> <p>
This doesn't have much to do with floodfill, but see This doesn't have much to do with floodfill, but see
the <a href="how_peerselection">peer selection page</a> the <a href="how_peerselection">peer selection page</a>
for a discussion of the vulnerabilities of peer selection for tunnels. for a discussion of the vulnerabilities of peer selection for tunnels.
</p>
<h2 id="history">History</h2>
<h2><a name="history">History</a></h2> <p>
<a href="netdb_discussion.html">Moved to the netdb disussion page</a>. <a href="netdb_discussion.html">Moved to the netdb disussion page</a>.
</p>
<h2><a name="future">Future Work</a></h2> <h2 id="future">Future Work</h2>
<p>
After network growth of 5x - 10x, there will be a significant chance of lookup failure due After network growth of 5x - 10x, there will be a significant chance of lookup failure due
to the O(1) lookup strategy, and implementation of an <i>iterative</i> lookup strategy will be required. to the O(1) lookup strategy, and implementation of an <i>iterative</i> lookup strategy will be required.
</p>
<p> <p>
End-to-end encryption of additional netDb lookups and responses. End-to-end encryption of additional netDb lookups and responses.
</p>
{% endblock %} {% endblock %}