* Reimplement the I2NP reading with less temporary memory allocation.
There is still significant GC churn, especially under load, but this
should help.
* Catch some oddball errors in the transport (message timeout while
establishing).
* Expire queued messages even when the writer is blocked.
* Reimplement most of the I2NP writing with less temporary memory
allocations (I2NP reading still gobbles memory).
* Don't go into a fast busy if an I2PTunnel 'server' is explicitly killed
(thanks mule!)
* Handle some more error conditions regarding abruptly closing sockets
(thanks Jonva!)
got some pretty heavy GC churn when under load. rough estimate is we allocate 5-8x as
much data as we need, copying it all over the place before forwarding it (or processing it).
this should cut down a few of those copies, but not enough yet. it'd be great to get that
down to 2x.
* lots of logging
up and debug in the new tcp transport, but it all works, and i dont like having big changes
sitting on my local machine (and there's no real need for branching atm)
2004-09-26 jrandom
* Complete rewrite of the TCP transport with IP autodetection and
low CPU overhead reconnections. More concise connectivity errors
are listed on the /oldconsole.jsp as well. The IP autodetection works
by listening to the first person who tells you what your IP address is
when you have not defined one yourself and you have no other TCP
connections.
* Update to the I2NP message format to add transparent verification at
the I2NP level (beyond standard TCP verification).
* Remove a potential weakness in our AESEngine's safeEncrypt and safeDecrypt
implementation (rather than verifying with E(H(key)), we now verify with
E(H(iv))).
* The above changes are NOT BACKWARDS COMPATIBLE.
* Removed all of the old unused PHTTP code.
* Refactor various methods and clean up some javadoc.
stack traces (rather than "oh, we're doing it when... uh... writing to the socket")
* increase the throttle max, since we want to be able to send a few concurrent
* detect situations where we may be inadvertantly flooding the netDb
and log them as CRIT with a stacktrace, as well as publish the count
of those events in the netDb
* detect potential netDb DoS situations by checking to see if we have
received more than 20 netDb lookups in 10 seconds, and if so,
probabalistically drop subsequent netDb messages (P=1-(10/numReceived)).
This is also published in the netDb.
* removed SourceRouteBlock & SourceRouteReplyMessage, as they're a redundant concept
that 1) takes up bandwidth 2) takes up CPU 3) smell funny.
now the TunnelCreateMessage includes a replyTag, replyKey, replyTunnel, and
replyGateway that they garlic encrypt their ACK/NACK through and with.
* tunnelCreateMessage doesn't need a seperate ACK - either we get a
TunnelCreateStatusMessage back or we don't.
* message structure mods for unique tunnel ID per hop (though currently all hops have
the same tunnel ID)
(making a searchReply message ~100 bytes, down from ~30KB, and the lookup message ~64 bytes, down from ~10KB)
* when we get the netDb searchReply or lookup message referencing someone we don't know,
we fire off a lookup for them
* reduced some excessive padding
* dropped the DbSearchReplyMessageHandler, since it shouldn't be used (all search replies
should be handled by a MessageSelector built by the original search message)
* removed some oddball constructors from the SendMessageDirectJob and SendTunnelMessageJob (always must specify a timeout)
* refactored SendTunnelMessageJob main handler method into smaller logical methods
the simple RouterThrottleImpl bases its decision entirely on how congested the jobQueue is - if there are jobs that have been waiting 5+ seconds, reject everything and stop reading from the network
(each i2npMessageReader randomly waits .5-1s when throttled before rechecking it)
minor adjustments in the stats published - removing a few useless ones and adding the router.throttleNetworkCause (which is the average ms lag in the jobQueue when an I2NP reader is throttled)
do i need to wrap the Input/Output streams we use to pipe data over the net with a verification wrapper for the messages?
e.g. prefix the serialization of all I2NPMessages sent on the wire with the SHA256 of that serialization and verify on read?
Ho hum, dunno. maybe its something else, but the ElG/AES+SessionTag already has integrity verification so the only thing I can
think of is a checksum error that got past TCP's checking and corrupted the AES stream.
* when allocating tunnels for a client, make sure it has a good amount of time left in it (using default values, this means at least 7.5 minutes)
* allow overriding the profile organizer's thresholds so as to enforce a minimum number of fast and reliable peers, allowing a base level of tunnel diversification. this is done through the "profileOrganizer.minFastPeers" router.config / context property (default minimum = 4 fast and reliable peers)
* don't be so harsh with the isFailing calculator regarding db lookup responses, since we've decreased the timeout. however, include "participated in a failed tunnel" as part of the criteria
* more logging than god
* for dropped messages, if it is a DeliveryStatusMessage its not an error, its just lag / congestion (keep the average delay as the new stat "inNetPool.droppedDeliveryStatusDelay")
* rather than have all jobs created hooked into the clock for offset updates, have the jobQueue stay hooked up and update any active jobs accordingly (killing a memory leak of a JobTiming objects - one per job)
* dont go totally insane during shutdown and log like mad (though the clientApp things still log like mad, since they don't know the router is going down)
* adjust memory buffer sizes based on real world values so we don't have to expand/contract a lot
* dont display things that are completely useless (who cares what the first 32 bytes of a public key are?)
* reduce temporary object creation
* use more efficient collections at times
* on shutdown, log some state information (ready/timed jobs, pending messages, etc)
* explicit GC every 10 jobs. yeah, not efficient, but just for now we'll keep 'er in there
* only reread the router config file if it changes (duh)
a rooted app context. The core itself has its own I2PAppContext
(see its javadoc for, uh, docs), and the router extends that to
expose the router's singletons. The main point of this is to
make it so that we can run multiple routers in the same JVM, even
to allow different apps in the same JVM to switch singleton
implementations (e.g. run some routers with one set of profile
calculators, and other routers with a different one).
There is still some work to be done regarding the actual boot up
of multiple routers in a JVM, as well as their configuration,
though the plan is to have the RouterContext override the
I2PAppContext's getProperty/getPropertyNames methods to read from
a config file (seperate ones per context) instead of using the
System.getProperty that the base I2PAppContext uses.
Once the multi-router is working, i'll shim in a VMCommSystem
that doesn't depend upon sockets or threads to read/write (and
that uses configurable message send delays / disconnects / etc,
perhaps using data from the routerContext.getProperty to drive it).
I could hold off until the sim is all working, but there's a
truckload of changes in here and I hate dealing with conflicts ;)
Everything works - I've been running 'er for a while and kicked
the tires a bit, but if you see something amiss, please let me
know.
