* Further preparation for removing I2CP crypto
* Added some validation to the DH key agreement (thanks $anon)
* Validate tunnel data message expirations (though not really a problem,
since tunnels expire)
* Minor PRNG threading cleanup
* Use a buffered PRNG, pulling the PRNG data off a larger precalculated
buffer, rather than the underlying PRNG's (likely small) one, which in
turn reduces the frequency of recalcing.
* More tuning to reduce temporary allocation churn
* Within the tunnel, use xor(IV, msg[0:16]) as the flag to detect dups,
rather than the IV by itself, preventing an attack that would let
colluding internal adversaries tag a message to determine that they are
in the same tunnel. Thanks dvorak for the catch!
* Drop long inactive profiles on startup and shutdown
* /configstats.jsp: web interface to pick what stats to log
* Deliver more session tags to account for wider window sizes
* Cache some intermediate values in our HMACSHA256 and BC's HMAC
* Track the client send rate (stream.sendBps and client.sendBpsRaw)
* UrlLauncher: adjust the browser selection order
* I2PAppContext: hooks for dummy HMACSHA256 and a weak PRNG
* StreamSinkClient: add support for sending an unlimited amount of data
* Migrate the tests out of the default build jars
2005-06-22 Comwiz
* Migrate the core tests to junit
* use the new raw i2np message format (the previous corruptions were due to above)
* add a new test component (UDPFlooder) which floods all peers at the rate desired
* packet munging fix for highly fragmented messages
* include basic slow start code
* fixed the UDP peer rate refilling
* cleaned up some nextSend scheduling
2005-03-11 jrandom
* Rather than the fixed resend timeout floor (10s), use 10s+RTT as the
minimum (increased on resends as before, of course).
* Always prod the clock update listeners, even if just to tell them that
the time hasn't changed much.
* Added support for explicit peer selection for individual tunnel pools,
which will be useful in debugging but not recommended for use by normal
end users.
* More aggressively search for the next hop's routerInfo on tunnel join.
* Give messages received via inbound tunnels that are bound to remote
locations sufficient time (taking into account clock skew).
* Give alternate direct send messages sufficient time (10s min, not 5s)
* Always give the end to end data message the explicit timeout (though the
old default was sufficient before)
* No need to give end to end messages an insane expiration (+2m), as we
are already handling skew on the receiving side.
* Don't complain too loudly about expired TunnelCreateMessages (at least,
not until after all those 0.5 and 0.5.0.1 users upgrade ;)
* Properly keep the sendBps stat
* When running the router with router.keepHistory=true, log more data to
messageHistory.txt
* Logging updates
* Minor formatting updates
* Only build failsafe tunnels if we need them
* Properly implement the selectNotFailingPeers so that we get a random
selection of peers, rather than using the strictOrdering (thanks dm!)
* Don't include too many "don't tell me about" peer references in the
lookup message - only send the 10 peer references closest to the target.
* (Merged the 0.5-pre branch back into CVS HEAD)
* Replaced the old tunnel routing crypto with the one specified in
router/doc/tunnel-alt.html, including updates to the web console to view
and tweak it.
* Provide the means for routers to reject tunnel requests with a wider
range of responses:
probabalistic rejection, due to approaching overload
transient rejection, due to temporary overload
bandwidth rejection, due to persistent bandwidth overload
critical rejection, due to general router fault (or imminent shutdown)
The different responses are factored into the profiles accordingly.
* Replaced the old I2CP tunnel related options (tunnels.depthInbound, etc)
with a series of new properties, relevent to the new tunnel routing code:
inbound.nickname (used on the console)
inbound.quantity (# of tunnels to use in any leaseSets)
inbound.backupQuantity (# of tunnels to keep in the ready)
inbound.length (# of remote peers in the tunnel)
inbound.lengthVariance (if > 0, permute the length by adding a random #
up to the variance. if < 0, permute the length
by adding or subtracting a random # up to the
variance)
outbound.* (same as the inbound, except for the, uh, outbound tunnels
in that client's pool)
There are other options, and more will be added later, but the above are
the most relevent ones.
* Replaced Jetty 4.2.21 with Jetty 5.1.2
* Compress all profile data on disk.
* Adjust the reseeding functionality to work even when the JVM's http proxy
is set.
* Enable a poor-man's interactive-flow in the streaming lib by choking the
max window size.
* Reduced the default streaming lib max message size to 16KB (though still
configurable by the user), also doubling the default maximum window
size.
* Replaced the RouterIdentity in a Lease with its SHA256 hash.
* Reduced the overall I2NP message checksum from a full 32 byte SHA256 to
the first byte of the SHA256.
* Added a new "netId" flag to let routers drop references to other routers
who we won't be able to talk to.
* Extended the timestamper to get a second (or third) opinion whenever it
wants to actually adjust the clock offset.
* Replaced that kludge of a timestamp I2NP message with a full blown
DateMessage.
* Substantial memory optimizations within the router and the SDK to reduce
GC churn. Client apps and the streaming libs have not been tuned,
however.
* More bugfixes thank you can shake a stick at.
2005-02-13 jrandom
* Updated jbigi source to handle 64bit CPUs. The bundled jbigi.jar still
only contains 32bit versions, so build your own, placing libjbigi.so in
your install dir if necessary. (thanks mule!)
* Added support for libjbigi-$os-athlon64 to NativeBigInteger and CPUID
(thanks spaetz!)
* Handle unexpected network read errors more carefully (thanks parg!)
* Added more methods to partially compare (DataHelper) and display
arrays (Base64.encode).
* Exposed the AES encryptBlock/decryptBlock on the context.aes()
* Be more generous on the throttle when just starting up the router
* Fix a missing scheduled event in the streaming lib (caused after reset)
* Add a new DisconnectListener on the I2PSocketManager to allow
notification of session destruction.
* Make sure our own router identity is valid, and if it isn't, build a new
one and restart the router. Alternately, you can run the Router with
the single command line argument "rebuild" and it will do the same.
* Fix for a long standing synchronization bug in the JobQueue (and added
some kooky flags to make sure it stays dead)
* Update the ministreaming lib to force mode=guaranteed if the default
lib is used, and mode=best_effort for all other libs.
* Fixed up the configuration overrides for the streaming socket lib
integration so that it properly honors env settings.
* More memory usage streamlining (last major revamp for now, i promise)
* Increase the tunnel test timeout rapidly if our tunnels are failing.
* Honor message expirations for some tunnel jobs that were prematurely
expired.
* Streamline memory usage with temporary object caches and more efficient
serialization for SHA256 calculation, logging, and both I2CP and I2NP
message handling.
* Fix some situations where we forward messages too eagerly. For a
request at the tunnel endpoint, if the tunnel is inbound and the target
is remote, honor the message by tunnel routing the data rather than
sending it directly to the requested location.
* Added a watchdog timer to do some baseline liveliness checking to help
debug some odd errors.
* Added a pair of summary stats for bandwidth usage, allowing easy export
with the other stats ("bw.sendBps" and "bw.receiveBps")
* Trimmed another memory allocation on message reception.
* Reimplement the I2NP reading with less temporary memory allocation.
There is still significant GC churn, especially under load, but this
should help.
* Catch some oddball errors in the transport (message timeout while
establishing).
* Expire queued messages even when the writer is blocked.
* Reimplement most of the I2NP writing with less temporary memory
allocations (I2NP reading still gobbles memory).
* Don't go into a fast busy if an I2PTunnel 'server' is explicitly killed
(thanks mule!)
* Handle some more error conditions regarding abruptly closing sockets
(thanks Jonva!)
got some pretty heavy GC churn when under load. rough estimate is we allocate 5-8x as
much data as we need, copying it all over the place before forwarding it (or processing it).
this should cut down a few of those copies, but not enough yet. it'd be great to get that
down to 2x.
* lots of logging
up and debug in the new tcp transport, but it all works, and i dont like having big changes
sitting on my local machine (and there's no real need for branching atm)
2004-09-26 jrandom
* Complete rewrite of the TCP transport with IP autodetection and
low CPU overhead reconnections. More concise connectivity errors
are listed on the /oldconsole.jsp as well. The IP autodetection works
by listening to the first person who tells you what your IP address is
when you have not defined one yourself and you have no other TCP
connections.
* Update to the I2NP message format to add transparent verification at
the I2NP level (beyond standard TCP verification).
* Remove a potential weakness in our AESEngine's safeEncrypt and safeDecrypt
implementation (rather than verifying with E(H(key)), we now verify with
E(H(iv))).
* The above changes are NOT BACKWARDS COMPATIBLE.
* Removed all of the old unused PHTTP code.
* Refactor various methods and clean up some javadoc.
stack traces (rather than "oh, we're doing it when... uh... writing to the socket")
* increase the throttle max, since we want to be able to send a few concurrent
* detect situations where we may be inadvertantly flooding the netDb
and log them as CRIT with a stacktrace, as well as publish the count
of those events in the netDb
* detect potential netDb DoS situations by checking to see if we have
received more than 20 netDb lookups in 10 seconds, and if so,
probabalistically drop subsequent netDb messages (P=1-(10/numReceived)).
This is also published in the netDb.
* removed SourceRouteBlock & SourceRouteReplyMessage, as they're a redundant concept
that 1) takes up bandwidth 2) takes up CPU 3) smell funny.
now the TunnelCreateMessage includes a replyTag, replyKey, replyTunnel, and
replyGateway that they garlic encrypt their ACK/NACK through and with.
* tunnelCreateMessage doesn't need a seperate ACK - either we get a
TunnelCreateStatusMessage back or we don't.
* message structure mods for unique tunnel ID per hop (though currently all hops have
the same tunnel ID)
(making a searchReply message ~100 bytes, down from ~30KB, and the lookup message ~64 bytes, down from ~10KB)
* when we get the netDb searchReply or lookup message referencing someone we don't know,
we fire off a lookup for them
* reduced some excessive padding
* dropped the DbSearchReplyMessageHandler, since it shouldn't be used (all search replies
should be handled by a MessageSelector built by the original search message)
* removed some oddball constructors from the SendMessageDirectJob and SendTunnelMessageJob (always must specify a timeout)
* refactored SendTunnelMessageJob main handler method into smaller logical methods
the simple RouterThrottleImpl bases its decision entirely on how congested the jobQueue is - if there are jobs that have been waiting 5+ seconds, reject everything and stop reading from the network
(each i2npMessageReader randomly waits .5-1s when throttled before rechecking it)
minor adjustments in the stats published - removing a few useless ones and adding the router.throttleNetworkCause (which is the average ms lag in the jobQueue when an I2NP reader is throttled)
do i need to wrap the Input/Output streams we use to pipe data over the net with a verification wrapper for the messages?
e.g. prefix the serialization of all I2NPMessages sent on the wire with the SHA256 of that serialization and verify on read?
Ho hum, dunno. maybe its something else, but the ElG/AES+SessionTag already has integrity verification so the only thing I can
think of is a checksum error that got past TCP's checking and corrupted the AES stream.
* when allocating tunnels for a client, make sure it has a good amount of time left in it (using default values, this means at least 7.5 minutes)
* allow overriding the profile organizer's thresholds so as to enforce a minimum number of fast and reliable peers, allowing a base level of tunnel diversification. this is done through the "profileOrganizer.minFastPeers" router.config / context property (default minimum = 4 fast and reliable peers)
* don't be so harsh with the isFailing calculator regarding db lookup responses, since we've decreased the timeout. however, include "participated in a failed tunnel" as part of the criteria
* more logging than god
* for dropped messages, if it is a DeliveryStatusMessage its not an error, its just lag / congestion (keep the average delay as the new stat "inNetPool.droppedDeliveryStatusDelay")
* rather than have all jobs created hooked into the clock for offset updates, have the jobQueue stay hooked up and update any active jobs accordingly (killing a memory leak of a JobTiming objects - one per job)
* dont go totally insane during shutdown and log like mad (though the clientApp things still log like mad, since they don't know the router is going down)
* adjust memory buffer sizes based on real world values so we don't have to expand/contract a lot
* dont display things that are completely useless (who cares what the first 32 bytes of a public key are?)
* reduce temporary object creation
* use more efficient collections at times
* on shutdown, log some state information (ready/timed jobs, pending messages, etc)
* explicit GC every 10 jobs. yeah, not efficient, but just for now we'll keep 'er in there
* only reread the router config file if it changes (duh)
a rooted app context. The core itself has its own I2PAppContext
(see its javadoc for, uh, docs), and the router extends that to
expose the router's singletons. The main point of this is to
make it so that we can run multiple routers in the same JVM, even
to allow different apps in the same JVM to switch singleton
implementations (e.g. run some routers with one set of profile
calculators, and other routers with a different one).
There is still some work to be done regarding the actual boot up
of multiple routers in a JVM, as well as their configuration,
though the plan is to have the RouterContext override the
I2PAppContext's getProperty/getPropertyNames methods to read from
a config file (seperate ones per context) instead of using the
System.getProperty that the base I2PAppContext uses.
Once the multi-router is working, i'll shim in a VMCommSystem
that doesn't depend upon sockets or threads to read/write (and
that uses configurable message send delays / disconnects / etc,
perhaps using data from the routerContext.getProperty to drive it).
I could hold off until the sim is all working, but there's a
truckload of changes in here and I hate dealing with conflicts ;)
Everything works - I've been running 'er for a while and kicked
the tires a bit, but if you see something amiss, please let me
know.
