15:26 < jrandom> 0) hi
15:26 < jrandom> 1) 0.6.1.7 and net status
15:26 < jrandom> 2) Experimental tunnel failures
15:26 < jrandom> 3) SSU and NATs
15:26 < jrandom> 4) Syndie
15:26 < jrandom> 5) ???
15:26 < jrandom> 0) hi
15:26 * jrandom waves
15:26 < jrandom> weekly status notes posted up at http://dev.i2p.net/pipermail/i2p/2005-December/001237.html
15:26 * ailouros read the notes
15:27 * jrandom is late, so I'll give y'all a moment to read up :)
15:29 < jrandom> ok, might as well jump on in to 1) 0.6.1.7 and net status
15:29 <@cervantes> *cough*
15:29 < jrandom> I don't have much more to add beyond whats in the mail on this point. anyone have any further comments/questions/ideas?
15:30 < Pseudonym> seems like doing performance optimization before changing the the tunnel creation algo might be backwards
15:30 < gott> I am getting a lot of "No HTTP method found in the request.
15:30 < gott> Software caused connection abort: socket write error
15:30 < gott> "
15:30 <@modulus> tunnel lag is much lower, i don't know if you made any changes or my ISP is better all of a suden.
15:30 < gott> from the I2PTunnel Webmanager
15:31 < jrandom> gott: those suggest bad http requests, or things that the eepproxy ouldn't understand
15:31 < jrandom> modulus: cool, we've been doing lots to try and improve things
15:31 < jrandom> Pseudonym: well, so far tunnel creation hasn't been our choke point - our choke point was much higher level stuff
15:32 < jrandom> otoh, the improvements of the last few revs have exposed some issues down there
15:32 < Pseudonym> oh, so the optimization has been related to other parts of the code?
15:32 < Pseudonym> cool
15:33 < jrandom> aye, at the SSU level, as well as the tunnel operation level. tunnel creation is not a performance sensitive operation [except when it is ;]
15:34 < jrandom> I'm doing some live net load testing though, gathering some non-anonymous load stats of different peers to try to narrow things down further
15:34 < ailouros> I wonder why sometimes I see more tunnels than those configured for a destination (eg. eeProxy, inbound 7 tunnels 4 outbound)
15:34 < jrandom> so, over the next few days when you see the router 7xgV transferring lots of data, well, dont mind it ;)
15:35 < jrandom> ailouros: when tunnel creation takes a while, it builds extras, just in case.
15:35 < jrandom> zzz outlines a few of the odd issues on that front too, and there's a patch being worked on to improve things a bit
15:35 < ailouros> I see.. but then why they all expire at the same time?
15:35 <@cervantes> jrandom: out of curiosity, when did you begin those tests?
15:35 < jrandom> cervantes: a few days ago
15:36 <@cervantes> ah cool, it's _not_ that then ;-)
15:36 < jrandom> dunno ailouros, depends on a few conditions. but there are some... *cough* oddities in the tunnel creation code, which I've been holding off messing with since its getting rewritten for 0.6.2
15:38 < ailouros> I see. I thought it was a policy matter... I'd rather see the tunnels die at different times unless there's a good reason not to
15:38 < ailouros> as in, tunnel creations are skewed
15:39 < jrandom> aye, there will be better randomization for 0.6.2, and zzz's patch adds some randomization for the current rev too
15:40 <+Complication> I wonder why an otherwise sane instance of i2phex... would decide to rehash files every other time I start it?
15:40 < jrandom> not a clue
15:40 <+Complication> Damaged configuration sounds the likely cause so far, but I've not deleted my config yet.
15:40 < jrandom> perhaps skewed timestamps?
15:42 <+Complication> Nah, they seem correct too
15:42 * jrandom knows not. never looked at that part of phex's cod
15:42 < jrandom> er, code
15:42 <+Complication> I'll see if deleting old config files does it any good
15:42 < jrandom> cool
15:43 < jrandom> ok, anything else on 1) Net status / 0.6.1.7?
15:43 < jrandom> if not, moving on to 2) Experimental tunnel failures
15:44 < jrandom> we've touched on this a bit already, and there's more in the notes and on zzz.i2p
15:44 < jrandom> zzz: do you have anything you want to add/bring up?
15:46 < jrandom> if not, lets move on to 3) SSU and NATs
15:46 < jrandom> bar: anything you want to add?
15:46 <+bar> nope, i have nothing else to add but what's in the mail
15:47 < jrandom> cool, yeah I've still got to reply to some of the details - i think our retransmission will already take care of some of the issues you bring up
15:48 < jrandom> the trick is going to be detecting which situation is in play, so we can automate the right procedure (or inform the user that they're screwed)
15:48 <+bar> all in due time, no hurry
15:49 <+bar> aye, i suggested a manual user setting to circumvent that problem for the time being, perhaps it's not possible, but we can discuss it later
15:50 < jrandom> yeah, manual overrides will help, but my experience with earlier i2p revs was that everyone (*everyone*) fucked it up ;) so automation is preferred
15:50 < jrandom> (everyone meaning myself included ;)
15:52 <+bar> agree
15:52 < ailouros> lol if I did too then there were something wrong with the docs, as I followed them bit by bit :D
15:53 <+bar> meanwhile, i will spend some time studying the peer testing
15:53 < jrandom> cool, thanks bar!
15:54 <+bar> (perhaps i could generate some useless spam regarding that as well :)
15:54 < jrandom> :)
15:55 < jrandom> ok, if there's nothing else on 3), lets move on to 4) Syndie
15:56 < jrandom> there has been a lot of progress on this front lately, with pretty substantial UI revamps since 0.6.1.7 came out
15:57 < jrandom> there's also a new standalone install/build, though since all of us have i2p installed, we don't need a separate one
15:57 < ailouros> I find that 6.1.7's layout is harder to use than 6.1.6's
15:58 < jrandom> hmm, are you running syndie in single user mode? and/or are you using the latest CVS build or the official 0.6.1.7 build?
15:58 < ailouros> official 0.6.1.7, single user
15:58 < jrandom> are you one of the proponents of the blog-like interface, as opposed to the threaded nav?
15:58 < ailouros> I am not, though I don't really know which is the blog-like
15:58 < ailouros> personally I'd rather have a threaded nav
15:59 < ailouros> (and some color-coding of new messages as well in thread view)
15:59 <+Complication> Relatively late CVS, single user
15:59 <+Complication> I've found a minor oddity (which I think, could be non-intended)
15:59 < jrandom> ah, there has been a lot of progress on that front in CVS ailouros
15:59 < ailouros> great :)
16:00 < jrandom> we've got a new threaded display too, using cervantes' suggested full traversal of just one branch, as opposed to all branches
16:00 <@cervantes> are those changes pushed to syndiemedia.i2p.net?
16:00 <+bla> Would it be a good idea to show some default examples for the location in http://localhost:7657/syndie/syndicate.jsp ?
16:00 < jrandom> syndiemedia.i2p.net is CVS head, yeah
16:00 <+Complication> When you've opened up a thread, and are currently reading its posts... and then choose to apply a filter to which no posts match (e.g. open thread "Syndie threading", apply filter "i2p.i2phex")...
16:00 < jrandom> aye, perhaps bla. new installs will have the three defaults in there, but examples would be good
16:01 <@cervantes> (the actual thread's tree needs to fully open too though)
16:01 <+Complication> ...it appears to leave the current posts displayed, as if they were matching or something...
16:01 <+Complication> Despite me definitely clicking the "Go" button.
16:01 <@cervantes> Complication: yeah I found that confusing too
16:02 < jrandom> hmm Complication, the general idea was to let you browse around while still looking at a post, but perhaps it'd be best to drop the posts being displayed
16:02 < jrandom> cervantes: ah, yeah expanding it to the leaf would be good, and should be trivial to do
16:02 <+Complication> Just noticed, and since it stuck out, thought I'd tell
16:02 <@cervantes> (or make it more obvious that there aren't any matches)
16:03 < jrandom> well, the thread nav says *no matches* :)
16:03 < ailouros> perhaps he's looking for a lighter
16:03 < jrandom> !thwap
16:03 <@cervantes> (or make it even more obvious that there aren't any matches)
16:03 < jrandom> <blink>No matches</blink>
16:03 <+Complication> Oops :)
16:04 < tethra> seems your !thwap got spaetz__ instead, jr!
16:04 <+Complication> Right, sometimes the thread navigator *does* feel a long distance away :)
16:04 < jrandom> yeah, we're experimenting with some css to float that down the side, as an option
16:05 <@cervantes> with skinning support you could have the thread top buttom left right etc
16:05 <@cervantes> ah as jr said
16:05 <+Complication> The "Threads" link takes one there fairly quick, though
16:05 <+Complication> ...if it's within the viewport currently.
16:06 <+Complication> And those who are used to keyboard-navigating can naturally press "End"
16:06 < jrandom> of course, this stuff is really simple to modify (as you can see from the rapid changes in CVS :), so if anyone has any suggestions (or mockups - html / png / etc), please, post 'em up whenever
16:07 < jrandom> I expect we'll have a main blog overview page in the next few days in cvs
16:08 < jrandom> ok, there's lots of other stuff going on w/ syndie, so swing on by http://localhost:7657/syndie/ for more info :)
16:08 < jrandom> anyone have anything else to bring up on that, or shall we move on to 5) ???
16:09 < zzz> hi just walked in. on 2), I'm looking for testers for my patch.
16:10 < zzz> My results are improvements in job lag and reliability, and reduction in router hangs. So hoping others will try.
16:10 < ailouros> that sounds good enough. what do I have to do?
16:11 < jrandom> heya zzz, ok cool, i'll be bashing it around a bit here too. its got lots of different components to it, so might be worth splitting up into pieces, but it does look good and is on track for 0.6.1.8
16:11 < ailouros> (average uptime is about 10h here :(
16:11 < zzz> If you have source code and ant just apply the patch - or I can put up an i2pupdate.zip if you want
16:12 < zzz> jrandom I'll work on splitting it up
16:12 < ailouros> I'll go for the update, thanks
16:13 < zzz> ailouros will put it on zzz.i2p within an hour - thanks
16:13 < jrandom> zzz: I wouldn't worry about it unless you've got spare time... I can read through the diff :)
16:13 < ailouros> thank you
16:14 < zzz> jrandom OK. There's some miscellaneous stuff that can easily be ripped out by either you or me.
16:16 < ailouros> I guess we're at 5) ??? now?
16:16 < zzz> jrandom other topic was Router OOMs with i2phex and possible SAM issues
16:16 < jrandom> aye ailouros
16:16 < jrandom> ah yeah zzz, it'd be great to track down whats up with SAM
16:17 < ailouros> j346, did you have the chance to check out my app?
16:17 < jrandom> what would Rule is if someone could jump on and take over maintenance of the SAM bridge, as I havent done any substantial work on it, and human hasn't been around for a while.
16:19 < jrandom> not yet ailouros, unfortunately. was a bit unsure about how it worked, so I've got to read through the source first
16:20 < ailouros> feel free to ask
16:20 < ailouros> (and good luck on the journey through the source, it's a good definition for the word "mess")
16:20 < jrandom> hehe
16:21 < zzz> correction my experience has been OOMs when using i2p-bt, not i2phex. Happens after about 24 hours when running one i2p-bt and in a few hours when running two i2p-bt
16:22 <+Complication> Mine happened after some late-night stress-testing.
16:22 <+Complication> (during which, let it be noted, I saw 5-minute averages of 50 KB/s)
16:22 < bar_> could you please remind me what your app is/does, ailouros? my memory is good but short...
16:22 <+Complication> Incoming, that is.
16:22 <+Complication> Outgoing was limited to 35 KB/s
16:22 <@cervantes> Complication: I've never heard it called late-night stress testing before...
16:22 < jrandom> nice Complication
16:23 <+Complication> cervantes: well, one *could* call it semi-daily megaleeching then :P
16:23 < ailouros> bar_: it's a working proof-of-concept for a distributed filesharing app which shares common blocks among differnt files (as suggested by polecat)
16:23 < bar_> ah, right, thanks ailouros
16:24 < tethra> cervantes: heheheh ;)
16:24 < ailouros> you're welcome (if anyone wants to get the source, it's in c/c++)
16:25 <+polecat> ailouros: Be careful, the chance of two binary blocks being the same is sufficiently rare, I'm mostly talking about pure theory that would be unuseful in practice.
16:25 < ailouros> polecat, I agree. My best guess is that it comes useful when you get different versions of the same files
16:25 < ailouros> like, a movie which has a corrupted block
16:25 <+polecat> You could transfer blocks of zeroes at lightning speeds! ("The next block is zeroes" "oh I have that already" "the next block is zeroes" "oh I have that already")
16:26 < ailouros> or an archive of other zip files
16:26 < jrandom> or e.g. modified ID3 tags, etc
16:26 < ailouros> exactly
16:26 <+polecat> True. But an easy way to "fix" a movie with a corrupted block is to tell bittorrent to download on top of it. Most clients will preserve the blocks whose hashes are the same, and overwrite the ones that are different.
16:26 < jrandom> archives of files probably won't work though, since they'd have to break on file boundaries
16:27 < ailouros> j636, that's why I want to implement LBFS's idea of splitting on data marks and not fixed block sizes
16:27 <@cervantes> the DC community used that method, by sharing file distributions in rarsets
16:27 <+polecat> What might be useful is to make a general binary error correction algorithm, then implement it on a huge scale. All blocks could be "corrected" into each other, and you'd only have to transmit the correction data, which might be smaller than transmitting the block itself.
16:29 <@cervantes> and then searches are basedon tiger hashes of those rar parts
16:29 <+Complication> Nice thought... sounds difficult though :)
16:29 <+polecat> But just a hash-for-hash equivalent... you'd never find two blocks alike!
16:29 < ailouros> cervantes, what's a "rarset"? :D (except a "RAR file", that is)
16:29 <+polecat> Unless both sides already had the file, one of them corrupted.
16:29 < ailouros> polecat, uh?
16:29 <@cervantes> ailouros: a split rar archive, with parity files if necessary
16:30 < ailouros> cervantes: I don't understand the advantage of doing that
16:31 <@cervantes> it's main benefit was to add pseudo-multi-source downloading to DC
16:32 < ailouros> well, that's part of the block sharing mechanism between files, isn't it?
16:34 < ailouros> polecat: about the bittorrent overwriting of damaged files, what it doesn't buy you is when you're trying to get multiple versions at once
16:35 <@cervantes> your client only matches/downloads valid parts, if you have parity files you can also recover damaged parts
16:35 < ailouros> with my system there are no damaged parts (files are assembled only when the composing blocks are downloaded and re-checked)
16:36 <@cervantes> stuff bittorrent does by default, except that you can't search specifically for individual parts
16:36 <+polecat> Multiple versions aren't likely to have a single bit in common though... which is why they're so stupid. Some jerk decides to re-encode the movie in postage stamp format, and gives it the same name.
16:37 <+polecat> Or another jerk takes random data and names it by the file you want to download.
16:37 < ailouros> lol that's correct
16:37 <@cervantes> exactly and rarset releases are immune to that...
16:37 < ailouros> but keep in mind that files from other networks (emule, kazaa, whatever) often come corrupted
16:38 <+polecat> rarset releases aren't immune...
16:38 <+polecat> You still have to figure out which rarset is the right one.
16:38 < ailouros> cervantes, how are rarsets immune to an idiot publishing random junk?
16:38 <@cervantes> (provided you have a reliable source)
16:39 <@cervantes> because a release group publishes hashes/distribution information
16:39 < ailouros> hahaha that's easy :D
16:39 <@cervantes> and stuff is marked as nuked if it's poor quality, folk remove it from shares
16:40 < ailouros> cervantes, that much my toy already does
16:40 <@cervantes> cool
16:40 < ailouros> you get the file descriptor from a trusted source, you multiget the file pronto
16:41 <@cervantes> sounds good ;-)
16:41 < ailouros> you don't get to sarch for files, but you can browse through each user's shared dire, so you can use a web crawler and cache the results
16:42 < ailouros> though I might add a search function sometime in the future if deemed necessary
16:44 < ailouros> I believe my toy, proprely developed into an app, can offer the caching and resiliancy the freenet people try to offer
16:44 < ailouros> as in static content distribution and caching
16:45 < ailouros> you read my blog, you cache it and offer it to other people when they want. you don't do anything more than leave the content there
16:45 < ailouros> don't like the content? delete it and we're all set
16:45 < jrandom> hmm, so do you see it as a backing store that could be used for syndie?
16:46 < ailouros> it CAN be used as a backing store
16:46 < ailouros> as it is now, you might even use it in place of jetty, in i2p default installations
16:46 < jrandom> e.g. attachments / links to [clunk hash="$foo"]my file[/clunk]
16:46 < ailouros> (well with a couple of minor changes :D )
16:46 < jrandom> heh
16:47 < jrandom> ok, yeah, I definitely don't understand how clunk works... wanna post about it in syndie, or put up an eepsite? :)
16:47 < ailouros> file hashes are downloaded on file request, and these hashes are automagically downloaded into the full file
16:48 < jrandom> right, but "down"loaded is a question of from where to where, etc. an overal network architecture description would be helpful
16:48 < ailouros> I'll write a decent doc first, then publish it somewhere
16:48 < jrandom> r0x0r, thanks
16:48 < ailouros> downloaded from wherever you got the hash from
16:48 < ailouros> plus everyone else sharing these blocks
16:49 < ailouros> think go!zilla and download accellerator :)
16:49 < jrandom> I think you misunderstand how much I am confused
16:49 < ailouros> but transparent and within i2p
16:49 < ailouros> lol guess so :D
16:50 < jrandom> a very, very basic explanation of e.g. "you run a clunk client, download from a clunk server, get info about clunk peers", etc
16:50 < jrandom> do I use a web browser to query a clunk client? or server? or peer?
16:51 < jrandom> (thats how lost I am)
16:51 < ailouros> redo from 0 :)
16:51 < ailouros> you use your web browser
16:51 < ailouros> you connect to your client
16:51 < ailouros> you browse others' dir with your browser
16:51 < ailouros> you select which files to download with your browser
16:51 < ailouros> your client does the dirty work
16:52 < ailouros> you get the downloaded file back
16:52 < ailouros> is this better? :)
16:52 < jrandom> ok great, thanks - so the "browse other's dir" is done by your client querying their client and responding back with an HTML representation of it
16:52 < ailouros> exactly
16:52 < jrandom> (or pulled from some server/superpeer/etc)
16:53 < jrandom> coo'
16:53 < ailouros> all the dirty work (finding duplicates, multidownloads and so on) is done by your (local) client transparently
16:54 < ailouros> what you see is, basically, a directory tree and some fiels you can download
16:54 < jrandom> cool
16:55 < ailouros> to publish your data you give away your public (p2p) address
16:55 < ailouros> and to share files you copy them (or symlink them) to the pub/ directory (or some subdir). It's that easy
16:57 * jrandom will dig through the source further, and look forward to more info :)
16:57 < jrandom> ok, anyone else have anything for the meeting?
16:57 < bar_> umm.. what's the difference between publishing and sharing, if i may ask? does publishing push the data to some datastore?
16:58 < ailouros> bar_: sharing is giving the blocks to download. publishing is letting the world know what you share
16:58 < ailouros> publishing is a subset of sharing
16:58 < bar_> aha, gotcha, thanks
16:58 < ailouros> for example, if you have half of a file, you share it but don't publish it
16:59 < jrandom> how would people know they ould get those blocks from you then?
16:59 < ailouros> and you have full control over which files you publish (unlike emule where every downloaded file is published)
16:59 < ailouros> because each client periodically sends information to the network about which blocks he has to offer
17:00 < jrandom> coo'
17:00 < ailouros> sends to the network as in server (as is now) or DHT (future)
17:00 < jrandom> so its mnet-esque, with a block tracker
17:00 < ailouros> err mnet-esque?
17:01 < jrandom> similar to how mnet (mnetproject.org) works
17:01 * ailouros is reading mnetproject.org
17:02 < ailouros> well, you have just your personal spaces, no shared spaces
17:02 < ailouros> and you don't PUSH blocks around
17:02 < jrandom> yeah, its not exactly the same as mnet, but it similar structurally
17:03 < jrandom> its like mnet where everyone is too broke to have anyone host their data ;)
17:03 < ailouros> yep
17:03 < ailouros> :D
17:03 < jrandom> ok, anyone else have anything else to bring up?
17:04 < jrandom> if not...
17:04 * jrandom winds up
17:04 * jrandom *baf*s the meeting closed