9957e6ef17keep track of how many messages are processed in the tunnel
jrandom
2004-06-20 01:09:04 +00:00
6a02c8383cthe data is hopefully discarded by now, so dont try to get at it
jrandom
2004-06-20 01:02:03 +00:00
bc0a4ee68ddiscard the data ASAP, and make sure we access only the safely cached snippets of it as necessary
jrandom
2004-06-20 00:59:33 +00:00
1ca615da77InNetMessage gets a context
jrandom
2004-06-20 00:57:02 +00:00
7da0cee29aturned BandwidthLimiter into an interface, removed some of its teeth, and cleaned up TrivialBandwidthLimiter
jrandom
2004-06-20 00:56:04 +00:00
f25bccd19fadd some stats for the simulator (data doesnt seem that interesting, so i havent moved them to the CommSystemImpl) use the .discardData() functionality
jrandom
2004-06-20 00:53:19 +00:00
4e5a2e012cupdate since bw limiter interface changed (but dont bother to use it anymore here) i wonder if i should remove the phttp transport now or keep it around in case it gets revived...
jrandom
2004-06-20 00:50:38 +00:00
c9ee2a92a3dont buffer the reads, since we dont want that buffer to interfere with either the bandwidth limiting or the AES decryption logging
jrandom
2004-06-20 00:46:57 +00:00
95a7938328reduce the max slice time (aka max time to pump out a message + some cleanup) to 60 seconds close connections to peers who are so slow that they leave messages on the queue to expire reduce the default max queue size per connection to 10 messages (as always, this is a configurable param, via "i2np.tcp.maxQueuedMessages" in router.config)
jrandom
2004-06-20 00:44:43 +00:00
baedcdb2c1handle situations where people dont specify a client name for a client app
jrandom
2004-06-20 00:40:16 +00:00
bc06b3671awhenever a tunnel completes, log how many messages we passed through it in the stats: tunnel.inboundMessagesProcessed tunnel.outboundMessagesProcessed tunnel.participatingMessagesProcessed (for the various tunnel types)
jrandom
2004-06-20 00:35:52 +00:00
a9172811careduce the grace period from 5 to 2 minutes, which will cause us to test peers more often also add some logging (log level == debug will display why we mark a peer as failing)
jrandom
2004-06-20 00:31:20 +00:00
91b1fd6d07InNetMessage now needs a reference to a context, so give it one
jrandom
2004-06-20 00:26:05 +00:00
bab7b8b9eddiscard the payload of a message ASAP (even though we may need to hang on to the message for a while, for its replyJob, etc) take note of the fact that the tunnel had activity minor logging and formatting updates
jrandom
2004-06-20 00:24:06 +00:00
1b7fb96ca8dont expose a method we dont need to expose
jrandom
2004-06-20 00:19:38 +00:00
6d84b8c02f1) use cachedXor to cut down on the, uh, xor-ing (which involves at least one new byte[32]) 2) implement an optimized 'should contain' algorithm, rather than being a wuss and building + comparing a BigInteger of the xor. 3) more unit tests this stuff is called a *lot*, since we need to pick what bucket things go in all the time.
jrandom
2004-06-20 00:18:28 +00:00
3e3749f011added some unit tests (adding the local key (delta == 0x00) and adding 1000 random keys, all making sure nothing b0rks)
jrandom
2004-06-20 00:13:05 +00:00
52384fb3a5persist the local router's info @ netDb/my.info in addition to under netDb/routerInfo-$hash.dat this makes it easy for harvesting with simulations
jrandom
2004-06-20 00:11:23 +00:00
e401670087i like logging, dont you like logging?
jrandom
2004-06-20 00:08:59 +00:00
ae0b4c59ccinclude the port # in the thread name, and logging
jrandom
2004-06-20 00:07:37 +00:00
0a8dc8afcclogging, its whats for dinner
jrandom
2004-06-20 00:06:33 +00:00
d59b94df66logging, deal with times when a client doesnt have a destination yet
jrandom
2004-06-20 00:05:30 +00:00
e28502454binclude the port in the thread name (useful for the sim)
jrandom
2004-06-20 00:03:45 +00:00
bbf73f0937enforce some sanity checks on the payload size. see recent rant in DatabaseSearchReplyMessage commit for why this is necessary.
jrandom
2004-06-19 23:58:24 +00:00
592519c45csomehow some people are getting situations where the payload doesnt decompress. wtf? do i need to wrap the Input/Output streams we use to pipe data over the net with a verification wrapper for the messages? e.g. prefix the serialization of all I2NPMessages sent on the wire with the SHA256 of that serialization and verify on read? Ho hum, dunno. maybe its something else, but the ElG/AES+SessionTag already has integrity verification so the only thing I can think of is a checksum error that got past TCP's checking and corrupted the AES stream.
jrandom
2004-06-19 23:56:41 +00:00
cc904ba9dcuse new SessionConfig constructor
jrandom
2004-06-19 23:46:08 +00:00
1679ba6719so this String.getBytes(), its inefficient? you don't say... (yeah, i know, this optimization takes advantage of the fact that the data in question uses single byte charsets)
jrandom
2004-06-19 23:41:55 +00:00
07fadd4a6cavoid string.getBytes like the plague
jrandom
2004-06-19 23:40:03 +00:00
57e1ff39e0new method: cachedXor which, suprisingly, determines the xor of a hash against another hash, caching up to a certain number of values currently uses an essentially random ejection policy, but this saves a lot of temporarly memory churn, since we xor many hashes against a router/destination's key
jrandom
2004-06-19 23:35:43 +00:00
51e259c198avoiding the String.getBytes() since its a bitch on gc (measured for this situation)
jrandom
2004-06-19 23:32:41 +00:00
de334b003dfor safety, always create a session config with a destination
jrandom
2004-06-19 23:30:57 +00:00
f4697be159just a simple catch all for OOM while handling an OOM (naw, we dont recurse too much (just a little))
jrandom
2004-06-19 23:25:47 +00:00
3835fe3960give the shutdown hook thread a name
jrandom
2004-06-19 23:13:09 +00:00
76c374ef06more accurate memory usage to reduce gc churn
jrandom
2004-06-19 23:11:42 +00:00
57d24bd948read the logger.config when its been updated, even if we dont have any log messages to write out (duh) also, a micro-optimization for charset handling identified through profiling
jrandom
2004-06-19 23:02:59 +00:00
ba6a2e3fd2unit test for the bandwidth limiting functionality (TrivialBandwidthLimiter, BandwidthLimiter, BandwidthLimited{In,Out}putStream)
jrandom
2004-06-13 20:03:21 +00:00
c3a395a41eupdate the bandwidth limiter config properties
jrandom
2004-06-13 19:59:44 +00:00
a3136a19e9big ol' rewrite, requiring new config settings and, er, it works pretty well. see router.config.template mods and the new unit tests. this implementation can cause starvation - e.g. lots of 1KB writes will go through before a 32KB write if the queue is low and the bwlimiter only replenishes say, 16KBps another impl would enforce a FIFO through thread wait/notify, etc, but would have the related overhead. i dont know whether starvation situations will be the norm or the exception. i'm running this on a few routers so we'll see.
jrandom
2004-06-13 19:56:57 +00:00
b631568003deal with null routers
jrandom
2004-06-13 19:48:23 +00:00
1d0c03eca4use the router context's properties (which now include the config settings)
jrandom
2004-06-13 19:47:44 +00:00
eb30525a26deal with null routers (useful for testing)
jrandom
2004-06-13 19:47:02 +00:00
3e66ea3f56include the router's config in the property settings
jrandom
2004-06-13 19:45:53 +00:00
9f1189e606use a bandwidth limited stream instead of asking for the allocation of the entire buffer at once (since, uh, its not likely that the bandwidth limiter will ever have hundreds of KBytes available for use)
jrandom
2004-06-13 19:39:42 +00:00
8fd02ee8ddallow the stream to optionally pull from the output stream's bandwidth limit queue (useful in very strange situations)
jrandom
2004-06-13 19:36:41 +00:00
f3154e8f5ebuffer between the bandwidth limiter and the raw stream
jrandom
2004-06-13 19:34:46 +00:00
878af163a9handle null boolean value (legal, but not in this context), fixes bug reported by nickster
jrandom
2004-06-13 19:32:58 +00:00
da8341d014reduced buffer size and updated domain name reference (dev.i2p.net, not i2p.dyndns.net)
jrandom
2004-06-13 19:31:22 +00:00
95c33e88edhandle decompress error by propogating the IOE (thanks nickster for bug report)
jrandom
2004-06-13 19:30:31 +00:00
fed8369a5fbig rewrite of how we load the native implementation: - the old method is default: it looks for jbigi.dll / libjbigi.so in the current dir (and library path) - otherwise, you can specify -Djbigi.impl=blah and if there is a file called "blah" in the classpath (including any jar file), it will load that file as a native implementation there are lots more javadocs now as well, and the dependency upon Log was removed (so anyone else who wants to use this just needs NativeBigInteger.java and the jbigi implementation)
jrandom
2004-06-12 20:08:32 +00:00
05918de6abfix bug #75 the databuffer is reused so it might contain junk, therefore only use the available amount of bytes (duck)
duck
2004-05-26 12:25:04 +00:00
7372ad0cc4A few changes to make proxy return the proper rfc2068 status codes instead of 404 for everything.
brianr
2004-05-24 07:08:02 +00:00
ca6884dbcaimports (sorry, includes alphabetizing, wee) (shendaras)
shendaras
2004-05-24 03:21:21 +00:00
1ebb0ac5fb0.3.1.4 (backwards compatible) i'll package & push later this evening
i2p_0_3_1_4
jrandom
2004-05-23 16:54:29 +00:00
bf0e53f13bi'll swallow your soul! er... make it queue up to 20 messages (in case of bursts), and do some more verbose logging
jrandom
2004-05-23 16:52:56 +00:00
04be41aac5if the send queue to the peer is too large, fail the message but also mark it as a comm error (since either their net con is insanely saturated, or disconnected) logging
jrandom
2004-05-22 12:05:34 +00:00
fd1313d49fbugger the speed estimate - always use the measured values, and if there aren't enough measured values, use 0.
jrandom
2004-05-22 12:03:38 +00:00
ddb6348bfdtype safety? we dont need no stinkin' type safety! (aka expiration vs timeoutMs, as shown by tunnel test messages that don't expire for another 30 years) this'll leak memory for failed tunnels, so we'll push this in a new rev in the next day or two, or maybe later today if it gets really bad.
jrandom
2004-05-20 13:43:09 +00:00
d70c5df5a00.3.1.3 (not backwards compatible, yadda yadda yadda)
i2p_0_3_1_3
jrandom
2004-05-20 11:32:32 +00:00
f2fa2038b1* made dbStore use a pessimistic algorithm - requiring confirmation of a store, rather than optimistically considering all store messages successful (NOT BACKWARDS COMPATIBLE) * when allocating tunnels for a client, make sure it has a good amount of time left in it (using default values, this means at least 7.5 minutes) * allow overriding the profile organizer's thresholds so as to enforce a minimum number of fast and reliable peers, allowing a base level of tunnel diversification. this is done through the "profileOrganizer.minFastPeers" router.config / context property (default minimum = 4 fast and reliable peers) * don't be so harsh with the isFailing calculator regarding db lookup responses, since we've decreased the timeout. however, include "participated in a failed tunnel" as part of the criteria * more logging than god * for dropped messages, if it is a DeliveryStatusMessage its not an error, its just lag / congestion (keep the average delay as the new stat "inNetPool.droppedDeliveryStatusDelay")
jrandom
2004-05-20 11:06:25 +00:00
bfd59e64earefactored the cleanup job logging
jrandom
2004-05-20 10:53:31 +00:00
60e05e270acleaned up slice processing reduced max queued messages per connection to 10 (additional ones are immediately marked as failed) update the I2P_FLAG byte to '*' making this NOT BACKWARDS COMPATIBLE formatting
jrandom
2004-05-20 10:51:22 +00:00
242b9a6af9fix the reread short circuiting algorithm
jrandom
2004-05-20 10:44:31 +00:00
e7e8ad9bdcadd the socketErrorListener interface (sorry duck)
jrandom
2004-05-19 22:30:52 +00:00
0e5d164a8asupport shutting down the router from the web console: specify a "router.shutdownPassword" value in the router.config (or in the environment [ala -D]), then specify that password on the shutdown form in the web console and hit submit. after 30 seconds, it'll kill the router (and unless you're running the sim, it'll kill the JVM too, including clientApp.* started tunnels / etc) if we had some sort of ACL for accessing the web console, we could avoid the password field altogether, but we dont, so we cant.
jrandom
2004-05-19 22:00:32 +00:00
7293a8d3c0more logging than your mom
jrandom
2004-05-19 15:24:25 +00:00
097a4647a8handle i2ptunnel server connection .accept()s asynchronously so we don't refuse lots of requests, causing intermittent "failures" use the new async error listening interface of the ministreaming lib truckloads of logging
jrandom
2004-05-19 15:20:55 +00:00
0942a7f3fftruckloads of logging new async interface for error notification (e.g. you can get notified of an error prior to it throwing the IOException). This async is useful since the IOException can be delayed for up to a minute while waiting for the close packet to be delivered. The alternative is to fire off a new thread to do the closing, and we may want to go there later, but i'm not sure.
jrandom
2004-05-19 15:14:30 +00:00
2df4370477Some changes to make the SAM module never block if called on a socket which select() says is safe to read/write or called in any case on a socket which is O_NONBLOCK
brianr
2004-05-19 01:26:02 +00:00
7243963106removed the insane explicit GC
jrandom
2004-05-18 18:39:43 +00:00
9f17654052tuned to avoid expensive biginteger operation (well, not "expensive", but a lot more expensive than a few shifts) when we need to debug, essentially assert the validity of the new tuned op, losing the performance benefit by duplicating the effort (to verify)
jrandom
2004-05-18 18:33:37 +00:00
292363eb65imports (sorry, includes alphabetizing, wee) (shendaras)
shendaras
2004-05-17 03:38:53 +00:00
07e79ce61a* do a db store after a successful db search (healing the netDb) * timeout each peer in a db search after 10 seconds, not 30 * logging
jrandom
2004-05-17 00:59:29 +00:00
ff0023a889big ol' memory, cpu usage, and shutdown handling update. main changes include: * rather than have all jobs created hooked into the clock for offset updates, have the jobQueue stay hooked up and update any active jobs accordingly (killing a memory leak of a JobTiming objects - one per job) * dont go totally insane during shutdown and log like mad (though the clientApp things still log like mad, since they don't know the router is going down) * adjust memory buffer sizes based on real world values so we don't have to expand/contract a lot * dont display things that are completely useless (who cares what the first 32 bytes of a public key are?) * reduce temporary object creation * use more efficient collections at times * on shutdown, log some state information (ready/timed jobs, pending messages, etc) * explicit GC every 10 jobs. yeah, not efficient, but just for now we'll keep 'er in there * only reread the router config file if it changes (duh)
jrandom
2004-05-16 04:54:50 +00:00
b0a1b3b5caadded some harvest options dont use javaw, since its a bitch to kill multiple jvms (yeah, this leaves a dos box. we'll deal until we've got the shutdown admin control)
jrandom
2004-05-13 23:32:57 +00:00
4c7af01edcallow dynamic update to the reliability threshold factor (e.g. rather than the top 2/3rds being considered "reliable", allow that to be the top 1/3, or 1/2, etc) the router.config var "profileOrganizer.reliabilityThresholdFactor=0.75" (or environment property -DprofileOrganizer.reliabilityThresholdFactor=0.5) etc
jrandom
2004-05-13 23:24:09 +00:00
0d431213cdinclude the previous period in the measurements (since they're discrete, not rolling) also include the other elements as necessary by default
jrandom
2004-05-13 07:14:54 +00:00
ad9dd9a2e2Lots of updates. I'm not calling this 0.3.1.2, still need to "burn it it" some more, but its looking good. * test all tunnels we manage every period or two. later we'll want to include some randomization to help fight traffic analysis, but that falls into the i2p 3.0 tunnel chaff / mixing / etc) * test inbound tunnels correctly (use an outbound tunnel, not direct) * only give the tunnels 30 seconds to succeed * mark the tunnel as tested for both the inbound and outbound side and adjust the profiles for all participants accordingly * keep track of the 'last test time' on a tunnel * new tunnel test response time profile stat, as well as overall router stat (published in the netDb as "tunnel.testSuccessTime") * rewrite of the speed calculator - the value it generates now is essentially "how many round trip messages can this router pass in a minute". it also allows a few runtime configurable options: = speedCalculator.eventThreshold: we use the smallest measurement period that has at least this many events in it (10m, 60m, 24h, lifetime) = speedCalculator.useInstantaneousRates: when we use the estimated round trip time, do we use instantaneous or period averages? = speedCalculator.useTunnelTestOnly: do we only use the tunnel test time (no db response or tunnel create time, or even estimated round trip times)? * fix the reliability calculator to use the 10 minute tunnel create successes, not the (almost always 0) 1 minute rate. * persist the tunnel create response time and send success time correctly (duh) * add a new main() to PeerProfile - PeerProfile [filename]* will calculate the values of the peer profiles specified. useful for tweaking the calculator, and/or the configurable options. ala: java -DspeedCalculator.useInstantaneousRates peerProfiles/profile-*.dat
jrandom
2004-05-13 04:32:26 +00:00
c7895ed905oh, you mean we're supposed to be at least a /little/ resiliant?
jrandom
2004-05-13 03:54:33 +00:00
jrandom