Added anonbib for use in papers list
Source: https://gitweb.torproject.org/anonbib.git Commit: b478fc493d4be2115185d94e077bf06196495417
This commit is contained in:
str4d
1268
i2p2www/anonbib/BibTeX.py
Normal file
1268
i2p2www/anonbib/BibTeX.py
Normal file
File diff suppressed because it is too large
Load Diff
39
i2p2www/anonbib/Makefile
Normal file
39
i2p2www/anonbib/Makefile
Normal file
@ -0,0 +1,39 @@
|
|||||||
|
PYTHON=python
|
||||||
|
VERSION=0.3-dev
|
||||||
|
|
||||||
|
all:
|
||||||
|
$(PYTHON) writeHTML.py anonbib.cfg
|
||||||
|
|
||||||
|
clean:
|
||||||
|
rm -f *~ */*~ *.pyc *.pyo
|
||||||
|
|
||||||
|
update:
|
||||||
|
$(PYTHON) updateCache.py anonbib.cfg
|
||||||
|
$(PYTHON) rank.py anonbib.cfg
|
||||||
|
|
||||||
|
suggest:
|
||||||
|
$(PYTHON) rank.py suggest anonbib.cfg
|
||||||
|
|
||||||
|
test:
|
||||||
|
$(PYTHON) test.py
|
||||||
|
|
||||||
|
veryclean: clean
|
||||||
|
rm -f author.html date.html topic.html bibtex.html tmp.bib
|
||||||
|
|
||||||
|
TEMPLATES=_template_.html _template_bibtex.html
|
||||||
|
CSS=css/main.css css/pubs.css
|
||||||
|
BIBTEX=anonbib.bib
|
||||||
|
SOURCE=BibTeX.py config.py metaphone.py reconcile.py updateCache.py \
|
||||||
|
writeHTML.py rank.py tests.py
|
||||||
|
EXTRAS=TODO README Makefile ChangeLog anonbib.cfg gold.gif silver.gif \
|
||||||
|
upb.gif ups.gif
|
||||||
|
|
||||||
|
DISTFILES=$(TEMPLATES) $(CSS) $(BIBTEX) $(SOURCE) $(EXTRAS)
|
||||||
|
|
||||||
|
dist: clean
|
||||||
|
rm -rf anonbib-$(VERSION)
|
||||||
|
mkdir anonbib-$(VERSION)
|
||||||
|
tar cf - $(DISTFILES) | (cd anonbib-$(VERSION); tar xf -)
|
||||||
|
mkdir anonbib-$(VERSION)/cache
|
||||||
|
tar czf anonbib-$(VERSION).tar.gz anonbib-$(VERSION)
|
||||||
|
rm -rf anonbib-$(VERSION)
|
||||||
52
i2p2www/anonbib/README
Normal file
52
i2p2www/anonbib/README
Normal file
@ -0,0 +1,52 @@
|
|||||||
|
anonbib 0.3 -- Code to generate the anonymity bibliography
|
||||||
|
|
||||||
|
Copyright (c) 2003-2008 Nick Mathewson
|
||||||
|
Based on 'PDOSBib' perl code by Eddie Kohler
|
||||||
|
|
||||||
|
This software is licensed under the GNU GPL, version 2 or later.
|
||||||
|
|
||||||
|
To use this software, you need to understand BibTeX and Python a
|
||||||
|
little. If it breaks, you get to keep both pieces. You will need
|
||||||
|
Python 2.2 or later.
|
||||||
|
|
||||||
|
To use this package:
|
||||||
|
- Get a good BibTeX file. You may want to mark it up with some of the
|
||||||
|
extra keys used in our "anonbib.bib" file. All of the additional
|
||||||
|
Bibtex keys we use have the prefix "www_"; check out anonbib.bib
|
||||||
|
for their usage.
|
||||||
|
|
||||||
|
- Edit anonbib.cfg and _template_.html and _template_bibtex.html so they
|
||||||
|
refer to your files, authors, topics, and so on.
|
||||||
|
|
||||||
|
- Run 'python updateCache.py anonbib.cfg' to create a local cache of the
|
||||||
|
papers in your bibliography based on their www_*_url entries. (By
|
||||||
|
default, the script will ignore any entries you have already cached. To
|
||||||
|
force a fresh download of a cached file, delete it.)
|
||||||
|
|
||||||
|
- Run 'python rank.py anonbib.cfg' to download Google Scholar rankings of
|
||||||
|
all the papers.
|
||||||
|
|
||||||
|
- Run 'python writeHTML.py anonbib.cfg'. Fix any errors you care about.
|
||||||
|
|
||||||
|
- Re-run these scripts when you change the bibliography.
|
||||||
|
|
||||||
|
- If you want to merge in big BibTeX files, try using the reconcile.py
|
||||||
|
script. See the comment at the start of the file for usage info.
|
||||||
|
|
||||||
|
|
||||||
|
New in 0.3:
|
||||||
|
- Support for Google Scholar rankings to denote hot/rising papers.
|
||||||
|
Implemented by George Danezis.
|
||||||
|
- Make reconcile script generate more useful output.
|
||||||
|
- Add support for multiple bibliographies generated from a single bibtex
|
||||||
|
source. This is done via 'tags' on bibtex entries. If an entry is
|
||||||
|
tagged, it appears in the corresponding bibliographies. This is good
|
||||||
|
for generating a master bibliography and one or more selected readings
|
||||||
|
lists from the same source.
|
||||||
|
- Handle more errors when downloading files.
|
||||||
|
- When fetching a paper with a .ps url, generate the .ps.gz file
|
||||||
|
automatically.
|
||||||
|
- Note an error when a crossref overrides an existing field in an entry.
|
||||||
|
- Handle the Proceedings type correctly.
|
||||||
|
- Enforce proper encoding on pages: it must be number--number.
|
||||||
|
-
|
||||||
33
i2p2www/anonbib/TODO
Normal file
33
i2p2www/anonbib/TODO
Normal file
@ -0,0 +1,33 @@
|
|||||||
|
|
||||||
|
|
||||||
|
- More general tasks
|
||||||
|
. Know about @book
|
||||||
|
. Write unit tests for everything
|
||||||
|
. Make name parsing vaguely sane
|
||||||
|
- Maybe uncrossref in tmp.bib
|
||||||
|
- Maybe pull important papers to the start of their sections?
|
||||||
|
. Clean \{}~ when going from note to url; add \{}~ when making
|
||||||
|
note from url.
|
||||||
|
. Also clean \_ to _ and back
|
||||||
|
- Look for urls in wherepublished.
|
||||||
|
. Forgive newlines in wherepublished, note.
|
||||||
|
- When sorting by date, entries with unknown months go into a magic
|
||||||
|
"month zero" before January. Is this right?
|
||||||
|
- Strip unused features.
|
||||||
|
o Take a configuration file on the command line instead of just
|
||||||
|
importing config.py.
|
||||||
|
|
||||||
|
- Cache tasks
|
||||||
|
- Generate a list of broken links
|
||||||
|
- Re-download all cached items if requested
|
||||||
|
- Clear dead items from cache
|
||||||
|
- Use HTTP HEAD requests to decide whetherto update stale
|
||||||
|
elements in cache.
|
||||||
|
- Add ability to honor a "www_no_cache={1}" option for entries
|
||||||
|
if the authors ask us not to cache them.
|
||||||
|
- Maybe, add ability to cache images from an HTML page.
|
||||||
|
|
||||||
|
- Reconcile tasks
|
||||||
|
- Document it.
|
||||||
|
- Notice when there is new or different information of certain kinds
|
||||||
|
(pages, dates, etc) in the new information.
|
||||||
74
i2p2www/anonbib/_template_.html
Normal file
74
i2p2www/anonbib/_template_.html
Normal file
@ -0,0 +1,74 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
||||||
|
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||||||
|
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
||||||
|
<head>
|
||||||
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
||||||
|
<meta http-equiv="Content-Style-Type" content="text/css" />
|
||||||
|
|
||||||
|
<!-- *** I AM MACHINE GENERATED! DO NOT EDIT ME!
|
||||||
|
*** EDIT THE .bib FILE or _template_.html INSTEAD!
|
||||||
|
|
||||||
|
Generated by `%(command_line)s'
|
||||||
|
(c) Eddie Kohler 1999-2000, Nick Mathewson 2003 -->
|
||||||
|
|
||||||
|
<!-- anon bib anonymity bibliography -->
|
||||||
|
<title>%(title)s</title>
|
||||||
|
|
||||||
|
<link rel="stylesheet" type="text/css" href="%(root)s/css/main.css" />
|
||||||
|
<link rel="stylesheet" type="text/css" href="%(root)s/css/pubs.css" />
|
||||||
|
|
||||||
|
</head>
|
||||||
|
<body bgcolor="#ffffff" text="#000000" link="#bb0000" vlink="#990099"
|
||||||
|
alink="#ff9900" >
|
||||||
|
|
||||||
|
<h1 align="center">%(short_title)s</h1>
|
||||||
|
<p align="center">%(otherbibs)s</p>
|
||||||
|
<p align="center">%(choices)s</p>
|
||||||
|
|
||||||
|
<!-- Table 1: contains the sidebar and the body. -->
|
||||||
|
<table cellspacing="3" cellpadding="0" border="0" align="center" width="100%%">
|
||||||
|
<tr valign="top">
|
||||||
|
<td>
|
||||||
|
|
||||||
|
<!-- Table 2: The sidebar-->
|
||||||
|
<table align="right" cellspacing="0" cellpadding="5" width="100"
|
||||||
|
class="sidebar">
|
||||||
|
<tr valign="top"><td><p class="l1"><strong>%(sectiontypes)s:</strong><br /></p>
|
||||||
|
%(sections)s
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
</table><!-- End of table 2 -->
|
||||||
|
</td>
|
||||||
|
|
||||||
|
<td width="85%%">
|
||||||
|
|
||||||
|
<h2>Publications by %(field)s</h2>
|
||||||
|
|
||||||
|
<ul class="sections">
|
||||||
|
%(entries)s
|
||||||
|
</ul>
|
||||||
|
|
||||||
|
</td>
|
||||||
|
|
||||||
|
<td width="5%%"><br /></td>
|
||||||
|
|
||||||
|
</tr>
|
||||||
|
</table><!-- End of table 1 -->
|
||||||
|
|
||||||
|
|
||||||
|
<p class="contact">Please send new or corrected entries to
|
||||||
|
<a name="contact">
|
||||||
|
<span class="email"><anonbib@free<!-- -->haven.net></span></a>.
|
||||||
|
<br />
|
||||||
|
If you can, please format them as BibTeX; see our
|
||||||
|
<a href="bibtex.html">BibTeX source page</a> for examples.<br />
|
||||||
|
Remember to include URLs if possible:
|
||||||
|
<a href="http://citeseer.ist.psu.edu/online-nature01/">offline papers are
|
||||||
|
less useful</a>.
|
||||||
|
</p>
|
||||||
|
<p>The source code to anonbib is now in Git. See <a href="https://gitweb.torproject.org/anonbib.git">the anonbib gitweb page</a> for more information.
|
||||||
|
</p>
|
||||||
|
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
28
i2p2www/anonbib/_template_bibtex.html
Normal file
28
i2p2www/anonbib/_template_bibtex.html
Normal file
@ -0,0 +1,28 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
||||||
|
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||||||
|
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
||||||
|
<head>
|
||||||
|
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />
|
||||||
|
<meta http-equiv="Content-Style-Type" content="text/css" />
|
||||||
|
|
||||||
|
<!-- *** I AM MACHINE GENERATED! DO NOT EDIT ME!
|
||||||
|
*** EDIT THE .bib FILE or _template_.html INSTEAD!
|
||||||
|
|
||||||
|
Generated by `%(command_line)s'
|
||||||
|
(c) Eddie Kohler 1999-2000, Nick Mathewson 2003 -->
|
||||||
|
|
||||||
|
<title>%(title)s: BibTeX</title>
|
||||||
|
|
||||||
|
<link rel="stylesheet" type="text/css" href="%(root)s/css/main.css" />
|
||||||
|
<link rel="stylesheet" type="text/css" href="%(root)s/css/pubs.css" />
|
||||||
|
|
||||||
|
</head>
|
||||||
|
<body bgcolor="#ffffff" text="#000000" link="#bb0000" vlink="#990099"
|
||||||
|
alink="#ff9900" >
|
||||||
|
|
||||||
|
<table cellspacing="15" border="0" align="center" width="100%%">
|
||||||
|
%(entries)s
|
||||||
|
</table>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
6293
i2p2www/anonbib/anonbib.bib
Normal file
6293
i2p2www/anonbib/anonbib.bib
Normal file
File diff suppressed because it is too large
Load Diff
163
i2p2www/anonbib/anonbib.cfg
Normal file
163
i2p2www/anonbib/anonbib.cfg
Normal file
@ -0,0 +1,163 @@
|
|||||||
|
# Copyright 2003-2008, Nick Mathewson. See LICENSE for licensing info.
|
||||||
|
|
||||||
|
# Our input filename.
|
||||||
|
MASTER_BIB = "./anonbib.bib"
|
||||||
|
|
||||||
|
# Where do we put generated HTML?
|
||||||
|
OUTPUT_DIR = "."
|
||||||
|
|
||||||
|
# Where do we put cached papers (relative to OUTPUT_DIR)
|
||||||
|
CACHE_DIR = "cache"
|
||||||
|
|
||||||
|
# Where do we cache citations papers (relative to OUTPUT_DIR)
|
||||||
|
CITE_CACHE_DIR = "cite_cache"
|
||||||
|
|
||||||
|
# Are there subsections for cached papers? This is useful for putting
|
||||||
|
# different Apache permission on different directories.
|
||||||
|
CACHE_SECTIONS = [ ]
|
||||||
|
|
||||||
|
# Only include entries that have this key. This is one way to
|
||||||
|
# generate multiple bibliographies from the same source. Currently
|
||||||
|
# deprecated in favor of tags.
|
||||||
|
#
|
||||||
|
#example: REQUIRE_KEY = "www_selected"
|
||||||
|
#
|
||||||
|
REQUIRE_KEY = None
|
||||||
|
|
||||||
|
# Timeout when downloading from a server while caching, in seconds.
|
||||||
|
DOWNLOAD_CONNECT_TIMEOUT = 15
|
||||||
|
|
||||||
|
# Template files.
|
||||||
|
TEMPLATE_FILE = "./_template_.html"
|
||||||
|
BIBTEX_TEMPLATE_FILE = "./_template_bibtex.html"
|
||||||
|
|
||||||
|
# Map from author name regex to author homepage.
|
||||||
|
AUTHOR_URLS = {
|
||||||
|
'Ross.*Anderson' : 'http://www.cl.cam.ac.uk/users/rja14/',
|
||||||
|
'Alessandro.*Acquisti' : 'http://www.heinz.cmu.edu/~acquisti/index.html',
|
||||||
|
'Agrawal' : 'http://www.research.ibm.com/people/a/agrawal/',
|
||||||
|
'Adam.*Back' : 'http://www.cypherspace.org/~adam/',
|
||||||
|
'Berthold' : 'http://page.inf.fu-berlin.de/~berthold/',
|
||||||
|
'Borisov' : 'http://hatswitch.org/~nikita/',
|
||||||
|
'Bettati' : 'http://faculty.cs.tamu.edu/bettati/',
|
||||||
|
'Miguel.*Castro' : 'http://research.microsoft.com/users/mcastro/',
|
||||||
|
'Chaum' : 'http://www.chaum.com/',
|
||||||
|
'J.*Claessens' : 'http://www.esat.kuleuven.be/~joclaess/',
|
||||||
|
'R.*Clayton' : 'http://www.cl.cam.ac.uk/~rnc1/',
|
||||||
|
'Wei Dai' : 'http://www.eskimo.com/~weidai/',
|
||||||
|
'Danezis' : 'http://homes.esat.kuleuven.be/~gdanezis/',
|
||||||
|
'Claudia.*az' : 'http://www.esat.kuleuven.be/~cdiaz/',
|
||||||
|
'Dingledine' : 'http://www.freehaven.net/~arma/cv.html',
|
||||||
|
'Desmedt' : 'http://www.cs.fsu.edu/~desmedt/',
|
||||||
|
'Douceur' : 'http://research.microsoft.com/~johndo/',
|
||||||
|
'N.*Hopper' : 'http://www-users.cs.umn.edu/~hopper/',
|
||||||
|
'Michael.*Freedman' : 'http://www.scs.cs.nyu.edu/~mfreed/',
|
||||||
|
'Gergely' : 'http://www.planeforge.com/home/tgm',
|
||||||
|
'Ian.*Goldberg' : 'http://www.cs.uwaterloo.ca/~iang/',
|
||||||
|
'Christian.*Grothoff' : 'http://grothoff.org/christian/',
|
||||||
|
'D.*Hopwood' : 'http://www.users.zetnet.co.uk/hopwood/',
|
||||||
|
'Jakobsson' : 'http://www2.parc.com/csl/members/mjakobss/markus-jakobsson.htm',
|
||||||
|
'Juels' : 'http://www.rsasecurity.com/rsalabs/staff/bios/ajuels/',
|
||||||
|
'Kaashoek' : 'http://pdos.csail.mit.edu/~kaashoek/',
|
||||||
|
'K.*Kurosawa' : 'http://kuro.cis.ibaraki.ac.jp/~kurosawa/',
|
||||||
|
'H.*Langos' : 'http://www.wh9.tu-dresden.de/~heinrich/',
|
||||||
|
'B.*Liskov' : 'http://www.pmg.lcs.mit.edu/barbara_liskov.html',
|
||||||
|
'Mathewson' : 'http://www.wangafu.net/~nickm/',
|
||||||
|
'Mazières' : 'http://www.scs.cs.nyu.edu/~dm/',
|
||||||
|
'B.*Möller' : ('http://www.informatik.tu-darmstadt.de/TI/'
|
||||||
|
'Mitarbeiter/moeller.html'),
|
||||||
|
'U.*Möller' : 'http://www.ulfm.de/',
|
||||||
|
'D.*Molnar' : 'http://www.cs.berkeley.edu/~dmolnar/',
|
||||||
|
'R.*Morris' : 'http://www.pdos.lcs.mit.edu/~rtm/',
|
||||||
|
'S.*Murdoch' : 'http://www.cl.cam.ac.uk/users/sjm217/',
|
||||||
|
'A.*Pashalidis' : 'http://www.xrtc.com/',
|
||||||
|
'A.*Pfitzmann' : 'http://dud.inf.tu-dresden.de/~pfitza/',
|
||||||
|
'B.*Pfitzmann' : 'http://www.zurich.ibm.com/~bpf/',
|
||||||
|
'B.*Preneel' : 'http://www.esat.kuleuven.be/~preneel/',
|
||||||
|
'Daniel.*Simon' : 'http://research.microsoft.com/crypto/dansimon/me.htm',
|
||||||
|
'Rackoff' : 'http://www.cs.toronto.edu/DCS/People/Faculty/rackoff.html',
|
||||||
|
'Jean F' : 'http://www.geocities.com/j_f_raymond/',
|
||||||
|
'M.*Rennhard' : 'http://www.tik.ee.ethz.ch/~rennhard/',
|
||||||
|
'M.*Reiter' : 'http://www.ece.cmu.edu/~reiter/',
|
||||||
|
'Rivest' : 'http://theory.lcs.mit.edu/~rivest/',
|
||||||
|
'Avi.*Rubin' : 'http://avirubin.com/',
|
||||||
|
'Sassaman' : 'http://homes.esat.kuleuven.be/~lsassama/',
|
||||||
|
'Serjantov' : 'http://home.arachsys.com/~aas/',
|
||||||
|
'S.*Seys' : 'http://www.esat.kuleuven.be/~sseys/',
|
||||||
|
'Shoup' : 'http://www.shoup.net/',
|
||||||
|
'Syverson' : 'http://www.syverson.org/',
|
||||||
|
'Tsudik' : 'http://www.ics.uci.edu/~gts/c.html',
|
||||||
|
'M.*Waidner' : 'http://www.zurich.ibm.com/~wmi/',
|
||||||
|
'David.*Wagner' : 'http://www.cs.berkeley.edu/~daw/',
|
||||||
|
'M.*Waldman' : 'http://cs1.cs.nyu.edu/~waldman/',
|
||||||
|
'B.*Waters' : 'http://www.cs.utexas.edu/~bwaters/',
|
||||||
|
'Chenxi.*Wang' : 'http://www.ece.cmu.edu/~chenxi/',
|
||||||
|
'M.*Wright' : 'http://ranger.uta.edu/~mwright/',
|
||||||
|
'B.*Levine' : 'http://prisms.cs.umass.edu/brian/',
|
||||||
|
'T.*Benjamin' : 'http://www.cs.umass.edu/~tshb/',
|
||||||
|
'B.*Defend' : 'http://www.cs.umass.edu/~defend/',
|
||||||
|
'K.*Fu' : 'http://www.cs.umass.edu/~kevinfu/',
|
||||||
|
'J.*Camenisch' : 'http://www.zurich.ibm.com/~jca/',
|
||||||
|
'S.*Hohenberger' : 'http://www.cs.jhu.edu/~susan/',
|
||||||
|
'M.*Kohlweiss' : 'http://homes.esat.kuleuven.be/~mkohlwei/',
|
||||||
|
'A.*Lysyanskaya' : 'http://www.cs.brown.edu/~anna/',
|
||||||
|
'M.*Meyerovich' : 'http://www.cs.brown.edu/~mira/',
|
||||||
|
'P.*Zieli.*ski' : 'http://www.cl.cam.ac.uk/~pz215/',
|
||||||
|
'S.*Zander' : 'http://caia.swin.edu.au/cv/szander/'
|
||||||
|
}
|
||||||
|
|
||||||
|
# List of paterns for author names _not_ to do an initial-tolerant
|
||||||
|
# match on when building section list. E.g., if "J\\. Smith" is in
|
||||||
|
# this list, he won't be folded into "John Smith".
|
||||||
|
NO_COLLAPSE_AUTHORS = [
|
||||||
|
|
||||||
|
]
|
||||||
|
|
||||||
|
# Map from LaTeX-style name of author to collapse to canonical name.
|
||||||
|
COLLAPSE_AUTHORS = {
|
||||||
|
"Nicholas Mathewson": "Nick Mathewson",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Map from author pattern to collation key.
|
||||||
|
# This keeps 'Zero Knowledge Systems' from getting alphabetized as "Systems,
|
||||||
|
# Zero Knowledge."
|
||||||
|
ALPHABETIZE_AUTHOR_AS = {
|
||||||
|
"Zero.*Knowledge.*Systems": "Zero Knowledge Systems",
|
||||||
|
"Carlos.*Aguilar.*Melchor": "Aguilar Melchor Carlos",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Map of strings to initialize BibTeX parsing with.
|
||||||
|
INITIAL_STRINGS = {
|
||||||
|
# SECTIONS
|
||||||
|
'sec_mix' : "Mix Networks: Design",
|
||||||
|
'sec_mixattacks' : "Mix Networks: Attacks",
|
||||||
|
'sec_stream' : "Stream-based anonymity",
|
||||||
|
'sec_traffic' : "Traffic analysis",
|
||||||
|
'sec_pub' : "Anonymous publication",
|
||||||
|
'sec_pir' : "Private Information Retrieval",
|
||||||
|
'sec_nym' : "Pseudonymity"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Don't put in any entries of this type.
|
||||||
|
OMIT_ENTRIES = ("proceedings", "journal")
|
||||||
|
|
||||||
|
# List of all recognized values for www_tags.
|
||||||
|
ALL_TAGS = ("selected", )
|
||||||
|
|
||||||
|
# Titles of page, by tag.
|
||||||
|
TAG_TITLES = { "": "The Free Haven Anonymity Bibliography",
|
||||||
|
"selected": "Free Haven's Selected Papers in Anonymity"
|
||||||
|
}
|
||||||
|
|
||||||
|
# As TAG_TITLES, but shorter.
|
||||||
|
TAG_SHORT_TITLES = { "": "Anonymity Bibliography",
|
||||||
|
"selected": "Selected Papers in Anonymity",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Directories where tag pages get generated.
|
||||||
|
TAG_DIRECTORIES = { '': "full",
|
||||||
|
"selected": "" }
|
||||||
|
|
||||||
|
# Make cached stuff group-writable. Make sure that your cache directories
|
||||||
|
# are sticky!
|
||||||
|
CACHE_UMASK = 002
|
||||||
56
i2p2www/anonbib/config.py
Normal file
56
i2p2www/anonbib/config.py
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
# Copyright 2003-2006, Nick Mathewson. See LICENSE for licensing info.
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
_KEYS = [ "ALL_TAGS",
|
||||||
|
"ALPHABETIZE_AUTHOR_AS","AUTHOR_URLS","CACHE_DIR","CACHE_SECTIONS",
|
||||||
|
"CACHE_UMASK",
|
||||||
|
"CITE_CACHE_DIR",
|
||||||
|
"COLLAPSE_AUTHORS",
|
||||||
|
"DOWNLOAD_CONNECT_TIMEOUT","INITIAL_STRINGS",
|
||||||
|
"MASTER_BIB", "NO_COLLAPSE_AUTHORS", "OMIT_ENTRIES",
|
||||||
|
"OUTPUT_DIR", "TEMPLATE_FILE", "BIBTEX_TEMPLATE_FILE",
|
||||||
|
"REQUIRE_KEY", "TAG_TITLES", "TAG_DIRECTORIES", "TAG_SHORT_TITLES",
|
||||||
|
]
|
||||||
|
|
||||||
|
for _k in _KEYS:
|
||||||
|
globals()[_k]=None
|
||||||
|
del _k
|
||||||
|
|
||||||
|
def load(cfgFile):
|
||||||
|
mod = {}
|
||||||
|
execfile(cfgFile, mod)
|
||||||
|
for _k in _KEYS:
|
||||||
|
try:
|
||||||
|
globals()[_k]=mod[_k]
|
||||||
|
except KeyError:
|
||||||
|
raise KeyError("Configuration option %s is missing"%_k)
|
||||||
|
|
||||||
|
INITIAL_STRINGS.update(_EXTRA_INITIAL_STRINGS)
|
||||||
|
AUTHOR_RE_LIST[:] = [
|
||||||
|
(re.compile(k, re.I), v,) for k, v in AUTHOR_URLS.items()
|
||||||
|
]
|
||||||
|
|
||||||
|
NO_COLLAPSE_AUTHORS_RE_LIST[:] = [
|
||||||
|
re.compile(pat, re.I) for pat in NO_COLLAPSE_AUTHORS
|
||||||
|
]
|
||||||
|
|
||||||
|
ALPHABETIZE_AUTHOR_AS_RE_LIST[:] = [
|
||||||
|
(re.compile(k, re.I), v,) for k,v in ALPHABETIZE_AUTHOR_AS.items()
|
||||||
|
]
|
||||||
|
|
||||||
|
_EXTRA_INITIAL_STRINGS = {
|
||||||
|
# MONTHS
|
||||||
|
'jan' : 'January', 'feb' : 'February',
|
||||||
|
'mar' : 'March', 'apr' : 'April',
|
||||||
|
'may' : 'May', 'jun' : 'June',
|
||||||
|
'jul' : 'July', 'aug' : 'August',
|
||||||
|
'sep' : 'September', 'oct' : 'October',
|
||||||
|
'nov' : 'November', 'dec' : 'December',
|
||||||
|
}
|
||||||
|
|
||||||
|
AUTHOR_RE_LIST = []
|
||||||
|
|
||||||
|
NO_COLLAPSE_AUTHORS_RE_LIST = []
|
||||||
|
|
||||||
|
ALPHABETIZE_AUTHOR_AS_RE_LIST = []
|
||||||
111
i2p2www/anonbib/css/main.css
Normal file
111
i2p2www/anonbib/css/main.css
Normal file
@ -0,0 +1,111 @@
|
|||||||
|
img {
|
||||||
|
border: 0px;
|
||||||
|
}
|
||||||
|
|
||||||
|
BODY {
|
||||||
|
background-color: #FFF;
|
||||||
|
color: #000;
|
||||||
|
margin: 0px;
|
||||||
|
}
|
||||||
|
|
||||||
|
FORM {
|
||||||
|
margin-top: 0.5em;
|
||||||
|
margin-bottom: 0.5em;
|
||||||
|
}
|
||||||
|
|
||||||
|
P, TD {
|
||||||
|
font-family: lucida, "Lucida Sans Unicode", Geneva, sans-serif;
|
||||||
|
}
|
||||||
|
|
||||||
|
P.contact {
|
||||||
|
text-align: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
P.contact A {
|
||||||
|
font-family: lucida, "Lucida Sans Unicode", Geneva, sans-serif;
|
||||||
|
font-weight: normal;
|
||||||
|
}
|
||||||
|
|
||||||
|
SPAN.email {
|
||||||
|
font-family: lucidatypewriter, "Lucida Typewriter", Monaco, "Lucida Sans Unicode", monospace;
|
||||||
|
font-weight: bold;
|
||||||
|
}
|
||||||
|
|
||||||
|
P IMG {
|
||||||
|
vertical-align: text-bottom;
|
||||||
|
}
|
||||||
|
|
||||||
|
P.crumbbreadth {
|
||||||
|
margin-top: 0.25em;
|
||||||
|
}
|
||||||
|
|
||||||
|
.compact {
|
||||||
|
margin-top: -0.5em;
|
||||||
|
text-indent: 0em;
|
||||||
|
}
|
||||||
|
|
||||||
|
SPAN.biblio {
|
||||||
|
font-style: italic;
|
||||||
|
}
|
||||||
|
|
||||||
|
SPAN.biblio A {
|
||||||
|
font-family: lucida, "Lucida Sans Unicode", Geneva, sans-serif;
|
||||||
|
font-weight: normal;
|
||||||
|
text-decoration: underline;
|
||||||
|
}
|
||||||
|
|
||||||
|
SPAN.availability {
|
||||||
|
font-family: lucidatypewriter, "Lucida Typewriter", Monaco, "Lucida Sans Unicode", monospace;
|
||||||
|
font-weight: normal;
|
||||||
|
}
|
||||||
|
|
||||||
|
UL {
|
||||||
|
list-style: outside;
|
||||||
|
}
|
||||||
|
|
||||||
|
UL.expand {
|
||||||
|
margin-bottom: 1em;
|
||||||
|
}
|
||||||
|
|
||||||
|
UL.sections {
|
||||||
|
list-style: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Font-level properties */
|
||||||
|
|
||||||
|
PRE {
|
||||||
|
font-family: lucidatypewriter, "Lucida Typewriter", Monaco, "Lucida Sans Unicode", monospace;
|
||||||
|
}
|
||||||
|
|
||||||
|
STRONG, A {
|
||||||
|
font-family: lucidatypewriter, "Lucida Typewriter", Monaco, "Rockwell", "Lucida Sans Unicode", monospace;
|
||||||
|
font-weight: bold;
|
||||||
|
}
|
||||||
|
|
||||||
|
A:link {
|
||||||
|
color: #B00;
|
||||||
|
}
|
||||||
|
|
||||||
|
A:visited {
|
||||||
|
color: #903;
|
||||||
|
}
|
||||||
|
|
||||||
|
H1, H2, H3, H4, H5, H6 {
|
||||||
|
font-family: lucidatypewriter, "Lucida Typewriter", "Lucida Console", Monaco, monospace;
|
||||||
|
}
|
||||||
|
|
||||||
|
H1 A, H2 A, H3 A, H4 A, H5 A, H6 A {
|
||||||
|
font-family: lucidatypewriter, "Lucida Typewriter", "Lucida Console", Monaco, monospace;
|
||||||
|
}
|
||||||
|
|
||||||
|
H1 {
|
||||||
|
color: #00B;
|
||||||
|
}
|
||||||
|
|
||||||
|
H2 {
|
||||||
|
color: #006;
|
||||||
|
}
|
||||||
|
|
||||||
|
H3 {
|
||||||
|
color: #006;
|
||||||
|
}
|
||||||
121
i2p2www/anonbib/css/pubs.css
Normal file
121
i2p2www/anonbib/css/pubs.css
Normal file
@ -0,0 +1,121 @@
|
|||||||
|
SPAN.title {
|
||||||
|
font-family: lucida, "Lucida Sans Unicode", Geneva, sans-serif;
|
||||||
|
font-weight: bold;
|
||||||
|
}
|
||||||
|
|
||||||
|
SPAN.author {
|
||||||
|
font-family: lucida, "Lucida Sans Unicode", Geneva, sans-serif;
|
||||||
|
font-weight: normal;
|
||||||
|
}
|
||||||
|
|
||||||
|
SPAN.availability {
|
||||||
|
font-family: lucidatypewriter, "Lucida Typewriter", Monaco, "Lucida Sans Unicode", monospace;
|
||||||
|
font-weight: normal;
|
||||||
|
}
|
||||||
|
|
||||||
|
SPAN.author A {
|
||||||
|
font-family: lucida, "Lucida Sans Unicode", Geneva, sans-serif;
|
||||||
|
font-weight: normal;
|
||||||
|
}
|
||||||
|
|
||||||
|
SPAN.biblio {
|
||||||
|
font-family: lucida, "Lucida Sans Unicode", Geneva, sans-serif;
|
||||||
|
font-style: italic;
|
||||||
|
}
|
||||||
|
|
||||||
|
SPAN.biblio A {
|
||||||
|
text-decoration: underline;
|
||||||
|
}
|
||||||
|
|
||||||
|
SPAN.bad {
|
||||||
|
text-decoration: underline;
|
||||||
|
color: #000;
|
||||||
|
background-color: #FDF;
|
||||||
|
}
|
||||||
|
|
||||||
|
P.remarks {
|
||||||
|
font-family: serif;
|
||||||
|
margin-top: 0.3em;
|
||||||
|
margin-bottom: 0;
|
||||||
|
margin-left: 5em;
|
||||||
|
padding-left: 0.5em;
|
||||||
|
border-width: 0 0 0 5px;
|
||||||
|
border-color: black;
|
||||||
|
border-style: solid;
|
||||||
|
}
|
||||||
|
|
||||||
|
P.remarks A {
|
||||||
|
text-decoration: underline;
|
||||||
|
}
|
||||||
|
|
||||||
|
P.l1 {
|
||||||
|
margin-left: 0.5em;
|
||||||
|
}
|
||||||
|
|
||||||
|
P.l2 {
|
||||||
|
margin-left: 1em;
|
||||||
|
margin-top: 0.3em;
|
||||||
|
margin-bottom: 0.3em;
|
||||||
|
}
|
||||||
|
|
||||||
|
P.entry {
|
||||||
|
margin-top: 0.7em;
|
||||||
|
margin-bottom: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
DIV.impEntry {
|
||||||
|
border-width: 1px;
|
||||||
|
border-color: black;
|
||||||
|
border-style: solid;
|
||||||
|
background-color: #FFE;
|
||||||
|
padding: 0.3em;
|
||||||
|
margin-top: 0.7em;
|
||||||
|
margin-bottom: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
P.impEntry {
|
||||||
|
background-color: #FFE;
|
||||||
|
padding: 0;
|
||||||
|
margin-top: 0;
|
||||||
|
margin-bottom: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
DIV.draftEntry {
|
||||||
|
/*
|
||||||
|
border-width: 1px;
|
||||||
|
border-color: black;
|
||||||
|
border-style: solid;
|
||||||
|
padding: 0.3em;
|
||||||
|
margin-top: 0.7em;
|
||||||
|
margin-bottom: 0;
|
||||||
|
*/
|
||||||
|
}
|
||||||
|
|
||||||
|
P.draftEntry {
|
||||||
|
color: #555;
|
||||||
|
padding: 0;
|
||||||
|
margin-top: 0;
|
||||||
|
margin-bottom: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
TABLE.sidebar {
|
||||||
|
border-width: 2px;
|
||||||
|
border-color: black;
|
||||||
|
border-style: solid;
|
||||||
|
background-color: #CFF;
|
||||||
|
}
|
||||||
|
|
||||||
|
TD.bibtex {
|
||||||
|
font-family: lucidatypewriter, "Lucida Typewriter", Monaco, "Lucida Sans Unicode", monospace;
|
||||||
|
border-width: 2px;
|
||||||
|
font-weight: normal;
|
||||||
|
border-color: black;
|
||||||
|
border-style: solid;
|
||||||
|
background-color: #DFF;
|
||||||
|
}
|
||||||
|
|
||||||
|
PRE.bibtex {
|
||||||
|
font-family: lucidatypewriter, "Lucida Typewriter", Monaco, "Lucida Sans Unicode", monospace;
|
||||||
|
font-size: smaller;
|
||||||
|
}
|
||||||
|
|
||||||
BIN
i2p2www/anonbib/gold.gif
Normal file
BIN
i2p2www/anonbib/gold.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 540 B |
193
i2p2www/anonbib/metaphone.py
Normal file
193
i2p2www/anonbib/metaphone.py
Normal file
@ -0,0 +1,193 @@
|
|||||||
|
#!/usr/bin/python2
|
||||||
|
# Copyright 2003-2008, Nick Mathewson. See LICENSE for licensing info.
|
||||||
|
|
||||||
|
"""metaphone.py -- Pure-python metaphone implementation.
|
||||||
|
|
||||||
|
(This is not guaranteed to match the real metaphone algorithm; I
|
||||||
|
haven't tested it thorougly enough. Let me know if you find bugs.
|
||||||
|
|
||||||
|
Based on the original C++ metaphone implementation.)
|
||||||
|
"""
|
||||||
|
|
||||||
|
TRIPLES = {
|
||||||
|
'dge': 'j',
|
||||||
|
'dgi': 'j',
|
||||||
|
'dgy': 'j',
|
||||||
|
'sia': '+x',
|
||||||
|
'sio': '+x',
|
||||||
|
'tia': '+x',
|
||||||
|
'tio': '+x',
|
||||||
|
'tch': '',
|
||||||
|
'tha': '0',
|
||||||
|
'the': '0',
|
||||||
|
'thi': '0',
|
||||||
|
'tho': '0',
|
||||||
|
'thu': '0',
|
||||||
|
}
|
||||||
|
|
||||||
|
DOUBLES = {
|
||||||
|
'ph' : 'f',
|
||||||
|
'sh' : 'x'
|
||||||
|
}
|
||||||
|
|
||||||
|
SINGLETONS = {
|
||||||
|
'd': 't',
|
||||||
|
'f': 'f',
|
||||||
|
'j': 'j',
|
||||||
|
'l': 'l',
|
||||||
|
'm': 'm',
|
||||||
|
'n': 'n',
|
||||||
|
'r': 'r',
|
||||||
|
'p': 'p',
|
||||||
|
'q': 'k',
|
||||||
|
'v': 'f',
|
||||||
|
'x': 'ks',
|
||||||
|
'z': 's',
|
||||||
|
}
|
||||||
|
|
||||||
|
ALLCHARS = "".join(map(chr, range(256)))
|
||||||
|
NONLCCHARS = "".join([c for c in ALLCHARS if not c.islower()])
|
||||||
|
def metaphone(s):
|
||||||
|
"""Return the metaphone equivalent of a provided string"""
|
||||||
|
s = s.lower()
|
||||||
|
s = s.translate(ALLCHARS, NONLCCHARS)
|
||||||
|
|
||||||
|
if not s: return ""
|
||||||
|
|
||||||
|
# If ae, gn, kn, pn, wr then drop the first letter.
|
||||||
|
if s[:2] in ("ae", "gn", "kn", "pn", "wr"):
|
||||||
|
s = s[1:]
|
||||||
|
|
||||||
|
# Change "x" to "s"
|
||||||
|
if s[0] == 'x':
|
||||||
|
s = "s%s" % s[1:]
|
||||||
|
|
||||||
|
# Get rid of "h" in "wh".
|
||||||
|
if s[:2] == 'wh':
|
||||||
|
s = "w%s" % s[1:]
|
||||||
|
|
||||||
|
# Get rid of s from end.
|
||||||
|
if s[-1] == 's':
|
||||||
|
s = s[:-1]
|
||||||
|
|
||||||
|
result = []
|
||||||
|
prevLtr = ' '
|
||||||
|
vowelBefore = 0
|
||||||
|
lastChar = len(s)-1
|
||||||
|
for idx in range(len(s)):
|
||||||
|
curLtr = s[idx]
|
||||||
|
# If first char is a vowel, keep it.
|
||||||
|
if curLtr in "aeiou":
|
||||||
|
if idx == 0:
|
||||||
|
result.append(curLtr)
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Skip double letters.
|
||||||
|
if idx < lastChar:
|
||||||
|
if curLtr == s[idx+1]:
|
||||||
|
continue
|
||||||
|
|
||||||
|
try:
|
||||||
|
r = TRIPLES[s[idx:idx+3]]
|
||||||
|
if r == "+x":
|
||||||
|
if idx > 1:
|
||||||
|
result.append("x")
|
||||||
|
continue
|
||||||
|
else:
|
||||||
|
result.append(r)
|
||||||
|
continue
|
||||||
|
except KeyError:
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
r = DOUBLES[s[idx:idx+2]]
|
||||||
|
result.append(r)
|
||||||
|
continue
|
||||||
|
except KeyError:
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
r = SINGLETONS[s[idx]]
|
||||||
|
result.append(r)
|
||||||
|
continue
|
||||||
|
except KeyError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
if idx > 0:
|
||||||
|
prevLtr = s[idx-1]
|
||||||
|
vowelBefore = prevLtr in "aeiou"
|
||||||
|
curLtr = s[idx]
|
||||||
|
|
||||||
|
nextLtr2 = ' '
|
||||||
|
if idx < lastChar:
|
||||||
|
nextLtr = s[idx+1]
|
||||||
|
vowelAfter = nextLtr in "aeiou"
|
||||||
|
frontvAfter = nextLtr in "eiy"
|
||||||
|
if idx+1 < lastChar:
|
||||||
|
nextLtr2 = s[idx+2]
|
||||||
|
else:
|
||||||
|
nextLtr = ' '
|
||||||
|
vowelAfter = frontvAfter = 0
|
||||||
|
|
||||||
|
|
||||||
|
if curLtr == 'b':
|
||||||
|
if idx == lastChar and prevLtr == 'm':
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
result.append(curLtr)
|
||||||
|
elif curLtr == 'c':
|
||||||
|
# silent 'sci', 'sce, 'scy', 'sci', etc OK.
|
||||||
|
if not (prevLtr == 's' and frontvAfter):
|
||||||
|
if nextLtr in 'ia':
|
||||||
|
result.append("x")
|
||||||
|
elif frontvAfter:
|
||||||
|
result.append("s")
|
||||||
|
elif prevLtr == 's' and nextLtr == 'h':
|
||||||
|
result.append('k')
|
||||||
|
elif nextLtr == 'h':
|
||||||
|
if idx == 0 and nextLtr2 in "aeiou":
|
||||||
|
result.append('k')
|
||||||
|
else:
|
||||||
|
result.append('x')
|
||||||
|
elif prevLtr == 'c':
|
||||||
|
result.append('c')
|
||||||
|
else:
|
||||||
|
result.append('k')
|
||||||
|
elif curLtr == 'g':
|
||||||
|
if (idx < lastChar-1) and nextLtr == 'h':
|
||||||
|
pass
|
||||||
|
elif s[idx:] == 'gned':
|
||||||
|
pass
|
||||||
|
elif s[idx:] == 'gn':
|
||||||
|
pass
|
||||||
|
elif prevLtr == 'd' and frontvAfter:
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
hard = (prevLtr == 'g')
|
||||||
|
if frontvAfter and not hard:
|
||||||
|
result.append('j')
|
||||||
|
else:
|
||||||
|
result.append('k')
|
||||||
|
elif curLtr == 'h':
|
||||||
|
if prevLtr in 'csptg':
|
||||||
|
pass
|
||||||
|
elif vowelBefore and not vowelAfter:
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
result.append('h')
|
||||||
|
elif curLtr == 'k':
|
||||||
|
if prevLtr != 'c': result.append('k')
|
||||||
|
elif curLtr in 'wy':
|
||||||
|
if vowelAfter:
|
||||||
|
result.append(curLtr)
|
||||||
|
|
||||||
|
return "".join(result)
|
||||||
|
|
||||||
|
def demo(a):
|
||||||
|
print a, "=>", metaphone(a)
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
demo("Nick. Mathewson")
|
||||||
|
|
||||||
|
demo("joe schmidt")
|
||||||
|
demo("Beethoven")
|
||||||
|
|
||||||
|
demo("Because the world is round")
|
||||||
202
i2p2www/anonbib/rank.py
Normal file
202
i2p2www/anonbib/rank.py
Normal file
@ -0,0 +1,202 @@
|
|||||||
|
# Make rankings of papers and authors for automatic classification of content hotness
|
||||||
|
|
||||||
|
# Google Scholar address
|
||||||
|
# http://scholar.google.com/scholar?as_epq=
|
||||||
|
|
||||||
|
# Take care of the caching setup
|
||||||
|
cache_expire = 60*60*24*30 # 30 days
|
||||||
|
|
||||||
|
# Checks
|
||||||
|
import config
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from os.path import exists, isdir, join, getmtime
|
||||||
|
from os import listdir, remove
|
||||||
|
|
||||||
|
def remove_old():
|
||||||
|
# Remove all old cached files
|
||||||
|
filenames = listdir(cache_folder())
|
||||||
|
from time import time
|
||||||
|
now = time()
|
||||||
|
for f in filenames:
|
||||||
|
pf = join(cache_folder(), f)
|
||||||
|
time_mt = getmtime(pf)
|
||||||
|
if now - time_mt > cache_expire: # 30 days
|
||||||
|
remove(pf)
|
||||||
|
|
||||||
|
def cache_folder():
|
||||||
|
r = join(config.OUTPUT_DIR, config.CITE_CACHE_DIR)
|
||||||
|
if not exists(r):
|
||||||
|
os.makedirs(r)
|
||||||
|
assert isdir(r)
|
||||||
|
return r
|
||||||
|
|
||||||
|
import re
|
||||||
|
from urllib2 import urlopen, build_opener
|
||||||
|
from urllib import quote
|
||||||
|
from datetime import date
|
||||||
|
import hashlib
|
||||||
|
|
||||||
|
# A more handy hash
|
||||||
|
def md5h(s):
|
||||||
|
m = hashlib.md5()
|
||||||
|
m.update(s)
|
||||||
|
return m.hexdigest()
|
||||||
|
|
||||||
|
format_tested = 0
|
||||||
|
|
||||||
|
def getPageForTitle(title, cache=True, update=True, save=True):
|
||||||
|
#Returns (citation-count, scholar url) tuple, or (None,None)
|
||||||
|
global format_tested
|
||||||
|
if not format_tested and update:
|
||||||
|
format_tested = 1
|
||||||
|
TestScholarFormat()
|
||||||
|
|
||||||
|
# Do not assume that the title is clean
|
||||||
|
title = re.sub("\s+", " ", title)
|
||||||
|
title = re.sub("[^'a-zA-Z0-9\. \-\/:]", "", title)
|
||||||
|
title = re.sub("'\/", " ", title)
|
||||||
|
|
||||||
|
# We rely on google scholar to return the article with this exact title
|
||||||
|
gurl = "http://scholar.google.com/scholar?as_q=&as_epq=%s&as_occt=title"
|
||||||
|
|
||||||
|
url = gurl % quote(title)
|
||||||
|
|
||||||
|
# Access cache or network
|
||||||
|
if exists(join(cache_folder(), md5h(url))) and cache:
|
||||||
|
return url, file(join(cache_folder(), md5h(url)),'r').read()
|
||||||
|
elif update:
|
||||||
|
print "Downloading rank for %r."%title
|
||||||
|
|
||||||
|
# Make a custom user agent (so that we are not filtered by Google)!
|
||||||
|
opener = build_opener()
|
||||||
|
opener.addheaders = [('User-agent', 'Anon.Bib.0.1')]
|
||||||
|
|
||||||
|
print "connecting..."
|
||||||
|
connection = opener.open(url)
|
||||||
|
print "reading"
|
||||||
|
page = connection.read()
|
||||||
|
print "done"
|
||||||
|
if save:
|
||||||
|
file(join(cache_folder(), md5h(url)),'w').write(page)
|
||||||
|
return url, page
|
||||||
|
else:
|
||||||
|
return url, None
|
||||||
|
|
||||||
|
def getCite(title, cache=True, update=True, save=True):
|
||||||
|
url, page = getPageForTitle(title, cache=cache, update=update, save=save)
|
||||||
|
if not page:
|
||||||
|
return None,None
|
||||||
|
|
||||||
|
# Check if it finds any articles
|
||||||
|
if len(re.findall("did not match any articles", page)) > 0:
|
||||||
|
return (None, None)
|
||||||
|
|
||||||
|
# Kill all tags!
|
||||||
|
cpage = re.sub("<[^>]*>", "", page)
|
||||||
|
|
||||||
|
# Add up all citations
|
||||||
|
s = sum([int(x) for x in re.findall("Cited by ([0-9]*)", cpage)])
|
||||||
|
return (s, url)
|
||||||
|
|
||||||
|
def getPaperURLs(title, cache=True, update=True, save=True):
|
||||||
|
url, page = getPageForTitle(title, cache=cache, update=update, save=save)
|
||||||
|
if not page:
|
||||||
|
return []
|
||||||
|
pages = re.findall(r'\&\#x25ba\;.*class=fl href="([^"]*)"', page)
|
||||||
|
return pages
|
||||||
|
|
||||||
|
def get_rank_html(title, years=None, base_url=".", update=True,
|
||||||
|
velocity=False):
|
||||||
|
s,url = getCite(title, update=update)
|
||||||
|
|
||||||
|
# Paper cannot be found
|
||||||
|
if s is None:
|
||||||
|
return ''
|
||||||
|
|
||||||
|
html = ''
|
||||||
|
|
||||||
|
url = url.replace("&","&")
|
||||||
|
|
||||||
|
# Hotness
|
||||||
|
H,h = 50,5
|
||||||
|
if s >= H:
|
||||||
|
html += '<a href="%s"><img src="%s/gold.gif" alt="More than %s citations on Google Scholar" title="More than %s citations on Google Scholar" /></a>' % (url,base_url,H,H)
|
||||||
|
elif s >= h:
|
||||||
|
html += '<a href="%s"><img src="%s/silver.gif" alt="More than %s citations on Google Scholar" title="More than %s citations on Google Scholar" /></a>' % (url,base_url,h,h)
|
||||||
|
|
||||||
|
# Only include the velocity if asked.
|
||||||
|
if velocity:
|
||||||
|
# Velocity
|
||||||
|
d = date.today().year - int(years)
|
||||||
|
if d >= 0:
|
||||||
|
if 2 < s / (d +1) < 10:
|
||||||
|
html += '<img src="%s/ups.gif" />' % base_url
|
||||||
|
if 10 <= s / (d +1):
|
||||||
|
html += '<img src="%s/upb.gif" />' % base_url
|
||||||
|
|
||||||
|
return html
|
||||||
|
|
||||||
|
def TestScholarFormat():
|
||||||
|
# We need to ensure that Google Scholar does not change its page format under our feet
|
||||||
|
# Use some cases to check if all is good
|
||||||
|
print "Checking google scholar formats..."
|
||||||
|
stopAndGoCites = getCite("Stop-and-Go MIXes: Providing Probabilistic Anonymity in an Open System", False)[0]
|
||||||
|
dragonCites = getCite("Mixes protected by Dragons and Pixies: an empirical study", False, save=False)[0]
|
||||||
|
|
||||||
|
if stopAndGoCites in (0, None):
|
||||||
|
print """OOPS.\n
|
||||||
|
It looks like Google Scholar changed their URL format or their output format.
|
||||||
|
I went to count the cites for the Stop-and-Go MIXes paper, and got nothing."""
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
if dragonCites != None:
|
||||||
|
print """OOPS.\n
|
||||||
|
It looks like Google Scholar changed their URL format or their output format.
|
||||||
|
I went to count the cites for a fictitious paper, and found some."""
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
def urlIsUseless(u):
|
||||||
|
if u.find("freehaven.net/anonbib/") >= 0:
|
||||||
|
# Our own cache is not the primary citation for anything.
|
||||||
|
return True
|
||||||
|
elif u.find("owens.mit.edu") >= 0:
|
||||||
|
# These citations only work for 'members of the MIT community'.
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
return False
|
||||||
|
|
||||||
|
URLTYPES=[ "pdf", "ps", "txt", "ps_gz", "html" ]
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
# First download the bibliography file.
|
||||||
|
import BibTeX
|
||||||
|
suggest = False
|
||||||
|
if sys.argv[1] == 'suggest':
|
||||||
|
suggest = True
|
||||||
|
del sys.argv[1]
|
||||||
|
|
||||||
|
config.load(sys.argv[1])
|
||||||
|
if config.CACHE_UMASK != None:
|
||||||
|
os.umask(config.CACHE_UMASK)
|
||||||
|
bib = BibTeX.parseFile(config.MASTER_BIB)
|
||||||
|
remove_old()
|
||||||
|
|
||||||
|
print "Downloading missing ranks."
|
||||||
|
for ent in bib.entries:
|
||||||
|
getCite(ent['title'], cache=True, update=True)
|
||||||
|
|
||||||
|
if suggest:
|
||||||
|
for ent in bib.entries:
|
||||||
|
haveOne = False
|
||||||
|
for utype in URLTYPES:
|
||||||
|
if ent.has_key("www_%s_url"%utype):
|
||||||
|
haveOne = True
|
||||||
|
break
|
||||||
|
if haveOne:
|
||||||
|
continue
|
||||||
|
print ent.key, "has no URLs given."
|
||||||
|
urls = [ u for u in getPaperURLs(ent['title']) if not urlIsUseless(u) ]
|
||||||
|
for u in urls:
|
||||||
|
print "\t", u
|
||||||
|
|
||||||
292
i2p2www/anonbib/reconcile.py
Normal file
292
i2p2www/anonbib/reconcile.py
Normal file
@ -0,0 +1,292 @@
|
|||||||
|
#!/usr/bin/python2
|
||||||
|
# Copyright 2003-2008, Nick Mathewson. See LICENSE for licensing info.
|
||||||
|
|
||||||
|
"""Code to determine which entries are new and which are old.
|
||||||
|
|
||||||
|
To scan a new file, run "python reconcile.py anonbib.cfg new-file.bib". This
|
||||||
|
will generate a new bibtex file called 'tmp.bib', with all the new entries
|
||||||
|
cleaned up a little, and all the duplicate entries commented out.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import re
|
||||||
|
|
||||||
|
assert sys.version_info[:3] >= (2,2,0)
|
||||||
|
|
||||||
|
import BibTeX
|
||||||
|
import config
|
||||||
|
import metaphone
|
||||||
|
|
||||||
|
_MPCACHE = {}
|
||||||
|
def soundsLike(s1, s2):
|
||||||
|
c = _MPCACHE
|
||||||
|
s1 = clean(s1)
|
||||||
|
s2 = clean(s2)
|
||||||
|
try:
|
||||||
|
m1 = c[s1]
|
||||||
|
except KeyError:
|
||||||
|
m1 = c[s1] = metaphone.metaphone(s1)
|
||||||
|
try:
|
||||||
|
m2 = c[s2]
|
||||||
|
except KeyError:
|
||||||
|
m2 = c[s2] = metaphone.metaphone(s2)
|
||||||
|
|
||||||
|
return m1 == m2
|
||||||
|
|
||||||
|
def mphone(s):
|
||||||
|
c = _MPCACHE
|
||||||
|
s = clean(s)
|
||||||
|
try:
|
||||||
|
return c[s]
|
||||||
|
except:
|
||||||
|
m = c[s] = metaphone.metaphone(s)
|
||||||
|
return m
|
||||||
|
|
||||||
|
def clean(s):
|
||||||
|
s = re.sub(r'\s+', ' ', s)
|
||||||
|
s = s.strip()
|
||||||
|
return s
|
||||||
|
|
||||||
|
class MasterBibTeX(BibTeX.BibTeX):
|
||||||
|
def __init__(self):
|
||||||
|
BibTeX.BibTeX.__init__(self)
|
||||||
|
|
||||||
|
def buildIndex(self):
|
||||||
|
self.byTitle = {}
|
||||||
|
for ent in self.entries:
|
||||||
|
for t in self._titleForms(ent['title']):
|
||||||
|
self.byTitle.setdefault(t, []).append(ent)
|
||||||
|
|
||||||
|
def _titleForms(self, title):
|
||||||
|
title = title.lower()
|
||||||
|
title = re.sub(r'\b(an|a|the|of)\b', "", title)
|
||||||
|
title = clean(title)
|
||||||
|
res = [ mphone(title) ]
|
||||||
|
if ':' in title:
|
||||||
|
for t in title.split(":"):
|
||||||
|
res.append(mphone(t.strip()))
|
||||||
|
#print "%r\n => %s" % (title,res)
|
||||||
|
return res
|
||||||
|
|
||||||
|
def _titlesAlike(self, t1, t2):
|
||||||
|
t1 = clean(t1)
|
||||||
|
t2 = clean(t2)
|
||||||
|
if t1 == t2:
|
||||||
|
return 2
|
||||||
|
tf1 = self._titleForms(t1)
|
||||||
|
tf2 = self._titleForms(t2)
|
||||||
|
for t in tf1:
|
||||||
|
if t in tf2: return 1
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def _authorsAlike(self, a1, a2):
|
||||||
|
if not soundsLike(" ".join(a1.last)," ".join(a2.last)):
|
||||||
|
return 0
|
||||||
|
|
||||||
|
if (a1.first == a2.first and a1.von == a2.von
|
||||||
|
and a1.jr == a2.jr):
|
||||||
|
return 2
|
||||||
|
|
||||||
|
|
||||||
|
if soundsLike(" ".join(a1.first), " ".join(a2.first)):
|
||||||
|
return 1
|
||||||
|
|
||||||
|
if not a1.first or not a2.first:
|
||||||
|
return 1
|
||||||
|
|
||||||
|
if self._initialize(a1.first) == self._initialize(a2.first):
|
||||||
|
return 1
|
||||||
|
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def _initialize(self, name):
|
||||||
|
name = " ".join(name).lower()
|
||||||
|
name = re.sub(r'([a-z])[a-z\.]*', r'\1', name)
|
||||||
|
name = clean(name)
|
||||||
|
return name
|
||||||
|
|
||||||
|
def _authorListsAlike(self, a1, a2):
|
||||||
|
if len(a1) != len(a2):
|
||||||
|
return 0
|
||||||
|
a1 = [ (a.last, a) for a in a1 ]
|
||||||
|
a2 = [ (a.last, a) for a in a2 ]
|
||||||
|
a1.sort()
|
||||||
|
a2.sort()
|
||||||
|
if len(a1) != len(a2):
|
||||||
|
return 0
|
||||||
|
r = 2
|
||||||
|
for (_, a1), (_, a2) in zip(a1,a2):
|
||||||
|
x = self._authorsAlike(a1,a2)
|
||||||
|
if not x:
|
||||||
|
return 0
|
||||||
|
elif x == 1:
|
||||||
|
r = 1
|
||||||
|
return r
|
||||||
|
|
||||||
|
def _entryDatesAlike(self, e1, e2):
|
||||||
|
try:
|
||||||
|
if clean(e1['year']) == clean(e2['year']):
|
||||||
|
return 2
|
||||||
|
else:
|
||||||
|
return 0
|
||||||
|
except KeyError:
|
||||||
|
return 1
|
||||||
|
|
||||||
|
def includes(self, ent, all=0):
|
||||||
|
title = ent['title']
|
||||||
|
candidates = []
|
||||||
|
for form in self._titleForms(title):
|
||||||
|
try:
|
||||||
|
candidates.extend(self.byTitle[form])
|
||||||
|
except KeyError:
|
||||||
|
pass
|
||||||
|
goodness = []
|
||||||
|
for knownEnt in candidates:
|
||||||
|
match = (self._entryDatesAlike(ent, knownEnt) *
|
||||||
|
self._titlesAlike(ent['title'], knownEnt['title']) *
|
||||||
|
self._authorListsAlike(ent.parsedAuthor,
|
||||||
|
knownEnt.parsedAuthor) )
|
||||||
|
if match:
|
||||||
|
goodness.append((match, knownEnt))
|
||||||
|
goodness.sort()
|
||||||
|
if all:
|
||||||
|
return goodness
|
||||||
|
if goodness:
|
||||||
|
return goodness[-1]
|
||||||
|
else:
|
||||||
|
return None, None
|
||||||
|
|
||||||
|
def demo(self):
|
||||||
|
for e in self.entries:
|
||||||
|
matches = self.includes(e, 1)
|
||||||
|
m2 = []
|
||||||
|
mids = []
|
||||||
|
for g,m in matches:
|
||||||
|
if id(m) not in mids:
|
||||||
|
mids.append(id(m))
|
||||||
|
m2.append((g,m))
|
||||||
|
matches = m2
|
||||||
|
|
||||||
|
if not matches:
|
||||||
|
print "No match for %s"%e.key
|
||||||
|
if matches[-1][1] is e:
|
||||||
|
print "%s matches for %s: OK."%(len(matches), e.key)
|
||||||
|
else:
|
||||||
|
print "%s matches for %s: %s is best!" %(len(matches), e.key,
|
||||||
|
matches[-1][1].key)
|
||||||
|
if len(matches) > 1:
|
||||||
|
for g, m in matches:
|
||||||
|
print "%%%% goodness", g
|
||||||
|
print m
|
||||||
|
|
||||||
|
|
||||||
|
def noteToURL(note):
|
||||||
|
" returns tp, url "
|
||||||
|
note = note.replace("\n", " ")
|
||||||
|
m = re.match(r'\s*(?:\\newline\s*)*\s*\\url{(.*)}\s*(?:\\newline\s*)*',
|
||||||
|
note)
|
||||||
|
if not m:
|
||||||
|
return None
|
||||||
|
url = m.group(1)
|
||||||
|
for suffix, tp in ((".html", "html"),
|
||||||
|
(".ps", "ps"),
|
||||||
|
(".ps.gz", "ps_gz"),
|
||||||
|
(".pdf", "pdf"),
|
||||||
|
(".txt", "txt")):
|
||||||
|
if url.endswith(suffix):
|
||||||
|
return tp,url
|
||||||
|
return "???", url
|
||||||
|
|
||||||
|
all_ok = 1
|
||||||
|
def emit(f,ent):
|
||||||
|
global all_ok
|
||||||
|
|
||||||
|
errs = ent._check()
|
||||||
|
if master.byKey.has_key(ent.key.strip().lower()):
|
||||||
|
errs.append("ERROR: Key collision with master file")
|
||||||
|
|
||||||
|
if errs:
|
||||||
|
all_ok = 0
|
||||||
|
|
||||||
|
note = ent.get("note")
|
||||||
|
if ent.getURL() and not note:
|
||||||
|
ent['note'] = "\url{%s}"%ent.getURL()
|
||||||
|
elif note:
|
||||||
|
m = re.match(r'\\url{(.*)}', note)
|
||||||
|
if m:
|
||||||
|
url = m.group(0)
|
||||||
|
tp = None
|
||||||
|
if url.endswith(".txt"):
|
||||||
|
tp = "txt"
|
||||||
|
elif url.endswith(".ps.gz"):
|
||||||
|
tp = "ps_gz"
|
||||||
|
elif url.endswith(".ps"):
|
||||||
|
tp = "ps_gz"
|
||||||
|
elif url.endswith(".pdf"):
|
||||||
|
tp = "pdf"
|
||||||
|
elif url.endswith(".html"):
|
||||||
|
tp = "html"
|
||||||
|
if tp:
|
||||||
|
ent['www_%s_url'%tp] = url
|
||||||
|
|
||||||
|
if errs:
|
||||||
|
all_ok = 0
|
||||||
|
for e in errs:
|
||||||
|
print >>f, "%%%%", e
|
||||||
|
|
||||||
|
print >>f, ent.format(77, 4, v=1, invStrings=invStrings)
|
||||||
|
|
||||||
|
def emitKnown(f, ent, matches):
|
||||||
|
print >>f, "%% Candidates are:", ", ".join([e.key for g,e in matches])
|
||||||
|
print >>f, "%%"
|
||||||
|
print >>f, "%"+(ent.format(77,4,1,invStrings).replace("\n", "\n%"))
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
if len(sys.argv) != 3:
|
||||||
|
print "reconcile.py expects 2 arguments"
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
config.load(sys.argv[1])
|
||||||
|
|
||||||
|
print "========= Scanning master =========="
|
||||||
|
master = MasterBibTeX()
|
||||||
|
master = BibTeX.parseFile(config.MASTER_BIB, result=master)
|
||||||
|
master.buildIndex()
|
||||||
|
|
||||||
|
print "========= Scanning new file ========"
|
||||||
|
try:
|
||||||
|
fn = sys.argv[2]
|
||||||
|
input = BibTeX.parseFile(fn)
|
||||||
|
except BibTeX.ParseError, e:
|
||||||
|
print "Error parsing %s: %s"%(fn,e)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
f = open('tmp.bib', 'w')
|
||||||
|
keys = input.newStrings.keys()
|
||||||
|
keys.sort()
|
||||||
|
for k in keys:
|
||||||
|
v = input.newStrings[k]
|
||||||
|
print >>f, "@string{%s = {%s}}"%(k,v)
|
||||||
|
|
||||||
|
invStrings = input.invStrings
|
||||||
|
|
||||||
|
for e in input.entries:
|
||||||
|
if not (e.get('title') and e.get('author')):
|
||||||
|
print >>f, "%%\n%%%% Not enough information to search for a match: need title and author.\n%%"
|
||||||
|
emit(f, e)
|
||||||
|
continue
|
||||||
|
|
||||||
|
matches = master.includes(e, all=1)
|
||||||
|
if not matches:
|
||||||
|
print >>f, "%%\n%%%% This entry is probably new: No match found.\n%%"
|
||||||
|
emit(f, e)
|
||||||
|
else:
|
||||||
|
print >>f, "%%"
|
||||||
|
print >>f, "%%%% Possible match found for this entry; max goodness",\
|
||||||
|
matches[-1][0], "\n%%"
|
||||||
|
emitKnown(f, e, matches)
|
||||||
|
|
||||||
|
if not all_ok:
|
||||||
|
print >>f, "\n\n\nErrors remain; not finished.\n"
|
||||||
|
|
||||||
|
f.close()
|
||||||
BIN
i2p2www/anonbib/silver.gif
Normal file
BIN
i2p2www/anonbib/silver.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 539 B |
1742
i2p2www/anonbib/testbib/pdos.bib
Normal file
1742
i2p2www/anonbib/testbib/pdos.bib
Normal file
File diff suppressed because it is too large
Load Diff
86
i2p2www/anonbib/tests.py
Normal file
86
i2p2www/anonbib/tests.py
Normal file
@ -0,0 +1,86 @@
|
|||||||
|
#!/usr/bin/python2
|
||||||
|
# Copyright 2004-2008, Nick Mathewson. See LICENSE for licensing info.
|
||||||
|
|
||||||
|
"""Unit tests for anonbib."""
|
||||||
|
|
||||||
|
import BibTeX
|
||||||
|
import metaphone
|
||||||
|
#import reconcile
|
||||||
|
#import writeHTML
|
||||||
|
#import updateCache
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
class MetaphoneTests(unittest.TestCase):
|
||||||
|
def testMetaphone(self):
|
||||||
|
pass
|
||||||
|
|
||||||
|
class BibTeXTests(unittest.TestCase):
|
||||||
|
def testTranslation(self):
|
||||||
|
ut = BibTeX.url_untranslate
|
||||||
|
self.assertEquals(ut("Fred"),"Fred")
|
||||||
|
self.assertEquals(ut("Hello, World."), "Hello_2c_20World.")
|
||||||
|
|
||||||
|
te = BibTeX.TeXescapeURL
|
||||||
|
ute = BibTeX.unTeXescapeURL
|
||||||
|
self.assertEquals(te("http://example/~me/my_file"),
|
||||||
|
r"http://example/\{}~me/my\_file")
|
||||||
|
self.assertEquals(ute(r"http:{}//example/\{}~me/my\_file"),
|
||||||
|
"http://example/~me/my_file")
|
||||||
|
|
||||||
|
h = BibTeX.htmlize
|
||||||
|
self.assertEquals(h("Hello, world"), "Hello, world")
|
||||||
|
self.assertEquals(h(r"\'a\`e\'{i}(\'\i)\"o&\^u"),
|
||||||
|
"áèí(í)ö&"
|
||||||
|
"û")
|
||||||
|
self.assertEquals(h(r"\~n and \c{c}"), "ñ and ç")
|
||||||
|
self.assertEquals(h(r"\AE---a ligature"), "Æ—a ligature")
|
||||||
|
self.assertEquals(h(r"{\it 33}"), " 33")
|
||||||
|
self.assertEquals(h(r"Pages 33--99 or vice--versa?"),
|
||||||
|
"Pages 33-99 or vice–versa?")
|
||||||
|
|
||||||
|
t = BibTeX.txtize
|
||||||
|
self.assertEquals(t("Hello, world"), "Hello, world")
|
||||||
|
self.assertEquals(t(r"\'a\`e\'{i}(\'\i)\"o&\^u"),
|
||||||
|
"aei(i)o&u")
|
||||||
|
self.assertEquals(t(r"\~n and \c{c}"), "n and c")
|
||||||
|
self.assertEquals(t(r"\AE---a ligature"), "AE---a ligature")
|
||||||
|
self.assertEquals(t(r"{\it 33}"), " 33")
|
||||||
|
self.assertEquals(t(r"Pages 33--99 or vice--versa?"),
|
||||||
|
"Pages 33--99 or vice--versa?")
|
||||||
|
|
||||||
|
def authorsParseTo(self,authors,result):
|
||||||
|
pa = BibTeX.parseAuthor(authors)
|
||||||
|
self.assertEquals(["|".join(["+".join(item) for item in
|
||||||
|
[a.first,a.von,a.last,a.jr]])
|
||||||
|
for a in pa],
|
||||||
|
result)
|
||||||
|
|
||||||
|
def testAuthorParsing(self):
|
||||||
|
pa = BibTeX.parseAuthor
|
||||||
|
PA = BibTeX.ParsedAuthor
|
||||||
|
apt = self.authorsParseTo
|
||||||
|
|
||||||
|
apt("Nick A. Mathewson and Roger Dingledine",
|
||||||
|
["Nick+A.||Mathewson|", "Roger||Dingledine|"])
|
||||||
|
apt("John van Neumann", ["John|van|Neumann|"])
|
||||||
|
apt("P. Q. Z. de la Paz", ["P.+Q.+Z.|de+la|Paz|"])
|
||||||
|
apt("Cher", ["||Cher|"])
|
||||||
|
apt("Smith, Bob", ["Bob||Smith|"])
|
||||||
|
apt("de Smith, Bob", ["Bob|de|Smith|"])
|
||||||
|
apt("de Smith, Bob Z", ["Bob+Z|de|Smith|"])
|
||||||
|
#XXXX Fix this.
|
||||||
|
#apt("Roberts Smith Wilkins, Bob Z", ["Bob+Z||Smith+Wilkins|"])
|
||||||
|
apt("Smith, Jr, Bob", ["Bob||Smith|Jr"])
|
||||||
|
|
||||||
|
#XXXX Fix this.
|
||||||
|
#apt("R Jones, Jr.", ["R||Jones|Jr."])
|
||||||
|
apt("Smith, Bob and John Smith and Last,First",
|
||||||
|
["Bob||Smith|", "John||Smith|", "First||Last|"])
|
||||||
|
apt("Bob Smith and John Smith and John Doe",
|
||||||
|
["Bob||Smith|", "John||Smith|", "John||Doe|"])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
unittest.main()
|
||||||
|
|
||||||
BIN
i2p2www/anonbib/upb.gif
Normal file
BIN
i2p2www/anonbib/upb.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 555 B |
169
i2p2www/anonbib/updateCache.py
Executable file
169
i2p2www/anonbib/updateCache.py
Executable file
@ -0,0 +1,169 @@
|
|||||||
|
#!/usr/bin/python
|
||||||
|
# Copyright 2003-2008, Nick Mathewson. See LICENSE for licensing info.
|
||||||
|
|
||||||
|
"""Download files in bibliography into a local cache.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import signal
|
||||||
|
import time
|
||||||
|
import gzip
|
||||||
|
|
||||||
|
import BibTeX
|
||||||
|
import config
|
||||||
|
import urllib2
|
||||||
|
import getopt
|
||||||
|
import socket
|
||||||
|
import errno
|
||||||
|
import httplib
|
||||||
|
|
||||||
|
FILE_TYPES = [ "txt", "html", "pdf", "ps", "ps.gz", "abstract" ]
|
||||||
|
BIN_FILE_TYPES = [ 'pdf', 'ps.gz' ]
|
||||||
|
|
||||||
|
class UIError(Exception):
|
||||||
|
pass
|
||||||
|
|
||||||
|
def tryUnlink(fn):
|
||||||
|
try:
|
||||||
|
os.unlink(fn)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def getCacheFname(key, ftype, section):
|
||||||
|
return BibTeX.smartJoin(config.OUTPUT_DIR,config.CACHE_DIR,
|
||||||
|
section,
|
||||||
|
"%s.%s"%(key,ftype))
|
||||||
|
|
||||||
|
def downloadFile(key, ftype, section, url,timeout=None):
|
||||||
|
if timeout is None:
|
||||||
|
timeout = config.DOWNLOAD_CONNECT_TIMEOUT
|
||||||
|
fname = getCacheFname(key, ftype, section)
|
||||||
|
parent = os.path.split(fname)[0]
|
||||||
|
if not os.path.exists(parent):
|
||||||
|
os.makedirs(parent)
|
||||||
|
|
||||||
|
fnameTmp = fname+".tmp"
|
||||||
|
fnameURL = fname+".url"
|
||||||
|
tryUnlink(fnameTmp)
|
||||||
|
|
||||||
|
def sigalrmHandler(sig,_):
|
||||||
|
pass
|
||||||
|
signal.signal(signal.SIGALRM, sigalrmHandler)
|
||||||
|
signal.alarm(timeout)
|
||||||
|
try:
|
||||||
|
try:
|
||||||
|
infile = urllib2.urlopen(url)
|
||||||
|
except httplib.InvalidURL, e:
|
||||||
|
raise UIError("Invalid URL %s: %s"%(url,e))
|
||||||
|
except IOError, e:
|
||||||
|
raise UIError("Cannot connect to url %s: %s"%(url,e))
|
||||||
|
except socket.error, e:
|
||||||
|
if getattr(e,"errno",-1) == errno.EINTR:
|
||||||
|
raise UIError("Connection timed out to url %s"%url)
|
||||||
|
else:
|
||||||
|
raise UIError("Error connecting to %s: %s"%(url, e))
|
||||||
|
finally:
|
||||||
|
signal.alarm(0)
|
||||||
|
|
||||||
|
mode = 'w'
|
||||||
|
if ftype in BIN_FILE_TYPES:
|
||||||
|
mode = 'wb'
|
||||||
|
outfile = open(fnameTmp, mode)
|
||||||
|
try:
|
||||||
|
while 1:
|
||||||
|
s = infile.read(1<<16)
|
||||||
|
if not s: break
|
||||||
|
outfile.write(s)
|
||||||
|
finally:
|
||||||
|
infile.close()
|
||||||
|
outfile.close()
|
||||||
|
|
||||||
|
urlfile = open(fnameURL, 'w')
|
||||||
|
print >>urlfile, time.strftime("%Y-%m-%d %H:%M:%S", time.localtime())
|
||||||
|
if "\n" in url: url = url.replace("\n", " ")
|
||||||
|
print >>urlfile, url
|
||||||
|
urlfile.close()
|
||||||
|
|
||||||
|
os.rename(fnameTmp, fname)
|
||||||
|
|
||||||
|
def getURLs(entry):
|
||||||
|
r = {}
|
||||||
|
for ftype in FILE_TYPES:
|
||||||
|
ftype2 = ftype.replace(".", "_")
|
||||||
|
url = entry.get("www_%s_url"%ftype2)
|
||||||
|
if url:
|
||||||
|
r[ftype] = url.strip().replace("\n", " ")
|
||||||
|
return r
|
||||||
|
|
||||||
|
def getCachedURL(key, ftype, section):
|
||||||
|
fname = getCacheFname(key, ftype, section)
|
||||||
|
urlFname = fname+".url"
|
||||||
|
if not os.path.exists(fname) or not os.path.exists(urlFname):
|
||||||
|
return None
|
||||||
|
f = open(urlFname, 'r')
|
||||||
|
lines = f.readlines()
|
||||||
|
f.close()
|
||||||
|
if len(lines) != 2:
|
||||||
|
print >>sys.stderr, "ERROR: unexpected number of lines in", urlFname
|
||||||
|
return lines[1].strip()
|
||||||
|
|
||||||
|
def downloadAll(bibtex, missingOnly=0):
|
||||||
|
"""returns list of tuples of key, ftype, url, error"""
|
||||||
|
errors = []
|
||||||
|
for e in bibtex.entries:
|
||||||
|
urls = getURLs(e)
|
||||||
|
key = e.key
|
||||||
|
section = e.get("www_cache_section", ".")
|
||||||
|
for ftype, url in urls.items():
|
||||||
|
if missingOnly:
|
||||||
|
cachedURL = getCachedURL(key, ftype, section)
|
||||||
|
if cachedURL == url:
|
||||||
|
print >>sys.stderr,"Skipping",url
|
||||||
|
continue
|
||||||
|
elif cachedURL is not None:
|
||||||
|
print >>sys.stderr,"URL for %s.%s has changed"%(key,ftype)
|
||||||
|
else:
|
||||||
|
print >>sys.stderr,"I have no copy of %s.%s"%(key,ftype)
|
||||||
|
try:
|
||||||
|
downloadFile(key, ftype, section, url)
|
||||||
|
print "Downloaded",url
|
||||||
|
except UIError, e:
|
||||||
|
print >>sys.stderr, str(e)
|
||||||
|
errors.append((key,ftype,url,str(e)))
|
||||||
|
except (IOError, socket.error), e:
|
||||||
|
msg = "Error downloading %s: %s"%(url,str(e))
|
||||||
|
print >>sys.stderr, msg
|
||||||
|
errors.append((key,ftype,url,msg))
|
||||||
|
if urls.has_key("ps") and not urls.has_key("ps.gz"):
|
||||||
|
# Say, this is something we'd like to have gzipped locally.
|
||||||
|
psFname = getCacheFname(key, "ps", section)
|
||||||
|
psGzFname = getCacheFname(key, "ps.gz", section)
|
||||||
|
if os.path.exists(psFname) and not os.path.exists(psGzFname):
|
||||||
|
# This is something we haven't gzipped yet.
|
||||||
|
print "Compressing a copy of",psFname
|
||||||
|
outf = gzip.GzipFile(psGzFname, "wb")
|
||||||
|
inf = open(psFname, "rb")
|
||||||
|
while 1:
|
||||||
|
s = inf.read(4096)
|
||||||
|
if not s:
|
||||||
|
break
|
||||||
|
outf.write(s)
|
||||||
|
outf.close()
|
||||||
|
inf.close()
|
||||||
|
|
||||||
|
return errors
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
if len(sys.argv) == 2:
|
||||||
|
print "Loading from %s"%sys.argv[1]
|
||||||
|
else:
|
||||||
|
print >>sys.stderr, "Expected a single configuration file as an argument"
|
||||||
|
sys.exit(1)
|
||||||
|
config.load(sys.argv[1])
|
||||||
|
|
||||||
|
if config.CACHE_UMASK != None:
|
||||||
|
os.umask(config.CACHE_UMASK)
|
||||||
|
|
||||||
|
bib = BibTeX.parseFile(config.MASTER_BIB)
|
||||||
|
downloadAll(bib,missingOnly=1)
|
||||||
BIN
i2p2www/anonbib/ups.gif
Normal file
BIN
i2p2www/anonbib/ups.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 536 B |
41
i2p2www/anonbib/venue-checklist.txt
Normal file
41
i2p2www/anonbib/venue-checklist.txt
Normal file
@ -0,0 +1,41 @@
|
|||||||
|
This file is to keep track of which volumes of which publications have
|
||||||
|
been combed for anonymity papers and which we still have to add.
|
||||||
|
|
||||||
|
=== DONE:
|
||||||
|
|
||||||
|
ExampleConference (through 2008)
|
||||||
|
|
||||||
|
PETS 2000-2003
|
||||||
|
|
||||||
|
=== CLAIMED:
|
||||||
|
|
||||||
|
PETS 2000-2010 -- Nick (claimed 6/16)
|
||||||
|
ESORICS 1990-2010 -- Nick (claimed 6/16)
|
||||||
|
CCS -- George (claimed 6/17)
|
||||||
|
USENIX Security ("Oakland") -- George (claimed 6/17)
|
||||||
|
|
||||||
|
=== SHOULD DO:
|
||||||
|
|
||||||
|
Infohiding
|
||||||
|
IEEE Security and privacy
|
||||||
|
NDSS
|
||||||
|
WPES
|
||||||
|
WEIS
|
||||||
|
Financial Crypto
|
||||||
|
Eurocrypt
|
||||||
|
Asiacrypt
|
||||||
|
|
||||||
|
Search: Papers that cite Chaum's paper
|
||||||
|
Search: Papers that cite the Tor paper
|
||||||
|
Search: Papers that cite the original onion routing papers
|
||||||
|
Search: Papers mentioning "anonymity" or "anonymous"
|
||||||
|
Search: Papers mentioning "mixnet" or "mix-net"
|
||||||
|
|
||||||
|
=== UNDERSERVED CONTENT; PLEASE SUGGEST SEARCHES AND VENUES
|
||||||
|
|
||||||
|
Private information retrieval; PIR
|
||||||
|
Anti-censorship; censorship
|
||||||
|
Location privacy
|
||||||
|
Anonymous credentials
|
||||||
|
Anonymizing data
|
||||||
|
Secure multiparty computation
|
||||||
246
i2p2www/anonbib/writeHTML.py
Executable file
246
i2p2www/anonbib/writeHTML.py
Executable file
@ -0,0 +1,246 @@
|
|||||||
|
#!/usr/bin/python
|
||||||
|
# Copyright 2003-2008, Nick Mathewson. See LICENSE for licensing info.
|
||||||
|
|
||||||
|
"""Generate indices by author, topic, date, and BibTeX key."""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import re
|
||||||
|
import os
|
||||||
|
import json
|
||||||
|
|
||||||
|
assert sys.version_info[:3] >= (2,2,0)
|
||||||
|
os.umask(022)
|
||||||
|
|
||||||
|
import BibTeX
|
||||||
|
import config
|
||||||
|
|
||||||
|
def getTemplate(name):
|
||||||
|
f = open(name)
|
||||||
|
template = f.read()
|
||||||
|
f.close()
|
||||||
|
template_s, template_e = template.split("%(entries)s")
|
||||||
|
return template_s, template_e
|
||||||
|
|
||||||
|
def pathLength(s):
|
||||||
|
n = 0
|
||||||
|
while s:
|
||||||
|
parent, leaf = os.path.split(s)
|
||||||
|
if leaf != '' and leaf != '.':
|
||||||
|
n += 1
|
||||||
|
s = parent
|
||||||
|
return n
|
||||||
|
|
||||||
|
def writeBody(f, sections, section_urls, cache_path, base_url):
|
||||||
|
'''f: an open file
|
||||||
|
sections: list of (sectionname, [list of BibTeXEntry])
|
||||||
|
section_urls: map from sectionname to external url'''
|
||||||
|
for s, entries in sections:
|
||||||
|
u = section_urls.get(s)
|
||||||
|
sDisp = re.sub(r'\s+', ' ', s.strip())
|
||||||
|
sDisp = sDisp.replace(" ", " ")
|
||||||
|
if u:
|
||||||
|
print >>f, ('<li><h3><a name="%s"></a><a href="%s">%s</a></h3>'%(
|
||||||
|
(BibTeX.url_untranslate(s), u, sDisp)))
|
||||||
|
else:
|
||||||
|
print >>f, ('<li><h3><a name="%s">%s</a></h3>'%(
|
||||||
|
BibTeX.url_untranslate(s),sDisp))
|
||||||
|
print >>f, "<ul class='expand'>"
|
||||||
|
for e in entries:
|
||||||
|
print >>f, e.to_html(cache_path=cache_path, base_url=base_url)
|
||||||
|
print >>f, "</ul></li>"
|
||||||
|
|
||||||
|
def writeHTML(f, sections, sectionType, fieldName, choices,
|
||||||
|
tag, config, cache_url_path, section_urls={}):
|
||||||
|
"""sections: list of (sectionname, [list of BibTeXEntry])'''
|
||||||
|
sectionType: str
|
||||||
|
fieldName: str
|
||||||
|
choices: list of (choice, url)"""
|
||||||
|
|
||||||
|
title = config.TAG_TITLES[tag]
|
||||||
|
short_title = config.TAG_SHORT_TITLES[tag]
|
||||||
|
#
|
||||||
|
secStr = []
|
||||||
|
for s, _ in sections:
|
||||||
|
hts = re.sub(r'\s+', ' ', s.strip())
|
||||||
|
hts = s.replace(" ", " ")
|
||||||
|
secStr.append("<p class='l2'><a href='#%s'>%s</a></p>\n"%
|
||||||
|
((BibTeX.url_untranslate(s),hts)))
|
||||||
|
secStr = "".join(secStr)
|
||||||
|
|
||||||
|
#
|
||||||
|
tagListStr = []
|
||||||
|
st = config.TAG_SHORT_TITLES.keys()
|
||||||
|
st.sort()
|
||||||
|
root = "../"*pathLength(config.TAG_DIRECTORIES[tag])
|
||||||
|
if root == "": root = "."
|
||||||
|
for t in st:
|
||||||
|
name = config.TAG_SHORT_TITLES[t]
|
||||||
|
if t == tag:
|
||||||
|
tagListStr.append(name)
|
||||||
|
else:
|
||||||
|
url = BibTeX.smartJoin(root, config.TAG_DIRECTORIES[t], "date.html")
|
||||||
|
tagListStr.append("<a href='%s'>%s</a>"%(url, name))
|
||||||
|
tagListStr = " | ".join(tagListStr)
|
||||||
|
|
||||||
|
#
|
||||||
|
choiceStr = []
|
||||||
|
for choice, url in choices:
|
||||||
|
if url:
|
||||||
|
choiceStr.append("<a href='%s'>%s</a>"%(url, choice))
|
||||||
|
else:
|
||||||
|
choiceStr.append(choice)
|
||||||
|
|
||||||
|
choiceStr = (" | ".join(choiceStr))
|
||||||
|
|
||||||
|
fields = { 'command_line' : "",
|
||||||
|
'sectiontypes' : sectionType,
|
||||||
|
'choices' : choiceStr,
|
||||||
|
'field': fieldName,
|
||||||
|
'sections' : secStr,
|
||||||
|
'otherbibs' : tagListStr,
|
||||||
|
'title': title,
|
||||||
|
'short_title': short_title,
|
||||||
|
"root" : root,
|
||||||
|
}
|
||||||
|
|
||||||
|
header, footer = getTemplate(config.TEMPLATE_FILE)
|
||||||
|
print >>f, header%fields
|
||||||
|
writeBody(f, sections, section_urls, cache_path=cache_url_path,
|
||||||
|
base_url=root)
|
||||||
|
print >>f, footer%fields
|
||||||
|
|
||||||
|
def jsonDumper(obj):
|
||||||
|
if isinstance(obj, BibTeX.BibTeXEntry):
|
||||||
|
e = obj.entries.copy()
|
||||||
|
e['key'] = obj.key
|
||||||
|
return e
|
||||||
|
else:
|
||||||
|
raise TypeError("Do not know how to serialize %s"%(obj.__class,))
|
||||||
|
|
||||||
|
def writePageSet(config, bib, tag):
|
||||||
|
if tag:
|
||||||
|
bib_entries = [ b for b in bib.entries
|
||||||
|
if tag in b.get('www_tags', "").split() ]
|
||||||
|
else:
|
||||||
|
bib_entries = bib.entries[:]
|
||||||
|
|
||||||
|
if not bib_entries:
|
||||||
|
print >>sys.stderr, "No entries with tag %r; skipping"%tag
|
||||||
|
return
|
||||||
|
|
||||||
|
tagdir = config.TAG_DIRECTORIES[tag]
|
||||||
|
outdir = os.path.join(config.OUTPUT_DIR, tagdir)
|
||||||
|
cache_url_path = BibTeX.smartJoin("../"*pathLength(tagdir),
|
||||||
|
config.CACHE_DIR)
|
||||||
|
if not os.path.exists(outdir):
|
||||||
|
os.makedirs(outdir, 0755)
|
||||||
|
##### Sorted views:
|
||||||
|
|
||||||
|
## By topic.
|
||||||
|
|
||||||
|
entries = BibTeX.sortEntriesBy(bib_entries, "www_section", "ZZZZZZZZZZZZZZ")
|
||||||
|
entries = BibTeX.splitSortedEntriesBy(entries, "www_section")
|
||||||
|
if entries[-1][0].startswith("<span class='bad'>"):
|
||||||
|
entries[-1] = ("Miscellaneous", entries[-1][1])
|
||||||
|
|
||||||
|
entries = [ (s, BibTeX.sortEntriesByDate(ents))
|
||||||
|
for s, ents in entries
|
||||||
|
]
|
||||||
|
|
||||||
|
f = open(os.path.join(outdir,"topic.html"), 'w')
|
||||||
|
writeHTML(f, entries, "Topics", "topic",
|
||||||
|
(("By topic", None),
|
||||||
|
("By date", "./date.html"),
|
||||||
|
("By author", "./author.html")
|
||||||
|
),
|
||||||
|
tag=tag, config=config,
|
||||||
|
cache_url_path=cache_url_path)
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
## By date.
|
||||||
|
|
||||||
|
entries = BibTeX.sortEntriesByDate(bib_entries)
|
||||||
|
entries = BibTeX.splitSortedEntriesBy(entries, 'year')
|
||||||
|
for idx in -1, -2:
|
||||||
|
if entries[idx][0].startswith("<span class='bad'>"):
|
||||||
|
entries[idx] = ("Unknown", entries[idx][1])
|
||||||
|
elif entries[idx][0].startswith("forthcoming"):
|
||||||
|
entries[idx] = ("Forthcoming", entries[idx][1])
|
||||||
|
sections = [ ent[0] for ent in entries ]
|
||||||
|
|
||||||
|
first_year = int(entries[0][1][0]['year'])
|
||||||
|
try:
|
||||||
|
last_year = int(entries[-1][1][0].get('year'))
|
||||||
|
except ValueError:
|
||||||
|
last_year = int(entries[-2][1][0].get('year'))
|
||||||
|
|
||||||
|
years = map(str, range(first_year, last_year+1))
|
||||||
|
if entries[-1][0] == 'Unknown':
|
||||||
|
years.append("Unknown")
|
||||||
|
|
||||||
|
f = open(os.path.join(outdir,"date.html"), 'w')
|
||||||
|
writeHTML(f, entries, "Years", "date",
|
||||||
|
(("By topic", "./topic.html"),
|
||||||
|
("By date", None),
|
||||||
|
("By author", "./author.html")
|
||||||
|
),
|
||||||
|
tag=tag, config=config,
|
||||||
|
cache_url_path=cache_url_path)
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
## By author
|
||||||
|
entries, url_map = BibTeX.splitEntriesByAuthor(bib_entries)
|
||||||
|
|
||||||
|
f = open(os.path.join(outdir,"author.html"), 'w')
|
||||||
|
writeHTML(f, entries, "Authors", "author",
|
||||||
|
(("By topic", "./topic.html"),
|
||||||
|
("By date", "./date.html"),
|
||||||
|
("By author", None),
|
||||||
|
),
|
||||||
|
tag=tag, config=config,
|
||||||
|
cache_url_path=cache_url_path,
|
||||||
|
section_urls=url_map)
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
## The big BibTeX file
|
||||||
|
|
||||||
|
entries = bib_entries[:]
|
||||||
|
entries = [ (ent.key, ent) for ent in entries ]
|
||||||
|
entries.sort()
|
||||||
|
entries = [ ent[1] for ent in entries ]
|
||||||
|
|
||||||
|
## Finding the root directory is done by writeHTML(), but
|
||||||
|
## the BibTeX file doesn't use that, so repeat the code here
|
||||||
|
root = "../"*pathLength(config.TAG_DIRECTORIES[tag])
|
||||||
|
if root == "": root = "."
|
||||||
|
|
||||||
|
header,footer = getTemplate(config.BIBTEX_TEMPLATE_FILE)
|
||||||
|
f = open(os.path.join(outdir,"bibtex.html"), 'w')
|
||||||
|
print >>f, header % { 'command_line' : "",
|
||||||
|
'title': config.TAG_TITLES[tag],
|
||||||
|
'root': root }
|
||||||
|
for ent in entries:
|
||||||
|
print >>f, (
|
||||||
|
("<tr><td class='bibtex'><a name='%s'>%s</a>"
|
||||||
|
"<pre class='bibtex'>%s</pre></td></tr>")
|
||||||
|
%(BibTeX.url_untranslate(ent.key), ent.key, ent.format(90,8,1)))
|
||||||
|
print >>f, footer
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
f = open(os.path.join(outdir,"bibtex.json"), 'w')
|
||||||
|
json.dump(entries, f, default=jsonDumper)
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
if len(sys.argv) == 2:
|
||||||
|
print "Loading from %s"%sys.argv[1]
|
||||||
|
else:
|
||||||
|
print >>sys.stderr, "Expected a single configuration file as an argument"
|
||||||
|
sys.exit(1)
|
||||||
|
config.load(sys.argv[1])
|
||||||
|
|
||||||
|
bib = BibTeX.parseFile(config.MASTER_BIB)
|
||||||
|
|
||||||
|
for tag in config.TAG_DIRECTORIES.keys():
|
||||||
|
writePageSet(config, bib, tag)
|
||||||
Reference in New Issue
Block a user