806 lines
36 KiB
HTML
806 lines
36 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
|
<html>
|
|
<head>
|
|
<title>Q User/Programmer Manual</title>
|
|
<style type="text/css">
|
|
<!--
|
|
td { vertical-align: top; }
|
|
code { font-family: courier, monospace; font-weight: bold }
|
|
-->
|
|
</style>
|
|
</head>
|
|
|
|
<body style="font-family: arial, helvetica, sans-serif">
|
|
<center>
|
|
<h1>Q User/Programmer Manual</h1>
|
|
|
|
<i>A brief but hopefully easy guide to installing and using the Q distributed file
|
|
store within the I2P network</i>
|
|
|
|
<br><br>
|
|
<i>(Return to <a href="../index.html">Q Homepage</a>)</i>
|
|
<br>
|
|
<br>
|
|
<small>
|
|
<a href="#intro">Introduction</a> |
|
|
<a href="#checklist">Checklist</a> |
|
|
<a href="#serverorclient">Server?orClient?</a> |
|
|
<a href="#walkthrough">Walkthrough</a> |
|
|
<a href="#server">Server Nodes</a> |
|
|
<a href="#qmgr">About QMgr</a> |
|
|
<a href="#contact">Contact us</a>
|
|
</small>
|
|
</center>
|
|
|
|
<a name="intro"/>
|
|
|
|
<hr>
|
|
|
|
<h2>1. Introduction</h2>
|
|
|
|
<blockquote>
|
|
Q is a distributed Peer2Peer file storage/retrieval network that aims to deliver optimal
|
|
performance by respecting the properties of the I2P network.<br>
|
|
<br>
|
|
This manual serves as a 'walkthrough' guide, to take you through the steps from initial
|
|
download, to everyday usage. It also provides information for the benefit of higher-level
|
|
client application authors.
|
|
</blockquote>
|
|
|
|
<a name="checklist"/>
|
|
|
|
<hr>
|
|
|
|
<h2>2. Preliminary Checklist</h2>
|
|
|
|
<blockquote>
|
|
OK, we assume here that you've already cracked the tarball, and are looking at
|
|
the distribution files.<br>
|
|
<br>
|
|
In order to get Q set up and running, you'll need:
|
|
<ol>
|
|
<li>An I2P router installed, set up and (permanently or transiently) running</li>
|
|
<li>Your system shell set up with at the environment variables:
|
|
<ul>
|
|
<li><b>CLASSPATH</b> - this should include:
|
|
<ul>
|
|
<li>The regular I2P jar files and 3rd party support jar files (eg <b>i2p.jar</b>,
|
|
<b>i2ptunnel.jar</b>, <b>streaming.jar</b>,
|
|
<b>mstreaming.jar</b>, <b>jbigi.jar</b>)</li>
|
|
<li>Apache's XML-RPC support jarfile - included in this Q distro as
|
|
<b>xmlrpc.jar</b></li>
|
|
<li>Aum's jarfile <b>aum.jar</b>, which includes Q and all needed support code</li>
|
|
</ul>
|
|
</li>
|
|
<li><b>PATH</b> - your execution search path <b><i>must</i></b> include the directory
|
|
in which your main java VM execution program (<b>java</b>, or on windows systems,
|
|
<b>java.exe</b>) resides.<br>
|
|
<b>NOTE</b> - if <b>java[.exe]</b> is not on your <b>PATH</b>, then Q <i>will
|
|
not run</i>.</li>
|
|
</ul>
|
|
</ol>
|
|
</blockquote>
|
|
|
|
<a name="serverorclient"/>
|
|
|
|
<hr>
|
|
|
|
<h2>3. Q Server or Q Client?</h2>
|
|
|
|
<blockquote>
|
|
Nearly everyone will want to run a <b>Q Client Node</b>.<br>
|
|
<Br>
|
|
It is only client nodes which provide users with full access to the Q network.<br>
|
|
<br>
|
|
However, if you have a (near-) permanently running I2P Router, and you're a kind and
|
|
generous soul, you might <i>also</i> be willing to run a <b>Q Server Node</b> in addition
|
|
to your <b>Q Client Node</b>.<br>
|
|
<br>
|
|
If you do choose to run a server node, you'll be expected to keep it running as near as
|
|
possible to 24/7. While transience of client nodes - frequent entering and leaving the
|
|
Q network - causes little or no disruption, transience of server nodes can significantly
|
|
impair Q's usability for everyone, particularly if this transience occurs frequently amongst
|
|
more than the smallest percentage of the server node pool.<br>
|
|
<br>
|
|
Until you're feeling well "settled in" with Q, your best approach is to just run a
|
|
client node for now, and add a server node later when you feel ready.<br>
|
|
</blockquote>
|
|
|
|
<a name="walkthrough"/>
|
|
|
|
<hr>
|
|
|
|
<h2>4. Q Walkthrough</h2>
|
|
|
|
<h3>4.1. Introduction</h3>
|
|
|
|
<blockquote>
|
|
This chapter discusses the deployment and usage of a Q Client Node, and will take you
|
|
through the steps of:
|
|
<ol>
|
|
<li>Double-checking that you've met the installation requirements</li>
|
|
<li>Launching a Q Client Node</li>
|
|
<li>Verifying that your Q Client Node is running</li>
|
|
<li>If your node fails to launch, figuring out why</li>
|
|
<li>Importing one or more noderefs into your node</li>
|
|
<li>Observing that your node is discovering other nodes on the network</li>
|
|
<li>Observing that your node is discovering content on the network</li>
|
|
<li>Searching for items of content that match chosen criteria</li>
|
|
<li>Retrieving stuff from the network</li>
|
|
<li>Inserting stuff to the network</li>
|
|
<li>Shutting down your client node</li>
|
|
</ol>
|
|
Setup and running of Q Server Nodes will be discussed in a later chapter.
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>4.2. Verify Your Q Installation Is Correct</h3>
|
|
|
|
<blockquote>
|
|
Ensure that all the needed I2P jarfiles, as well as <b>xmlrpc.jar</b> and
|
|
Q's very own <b>aum.jar</b> are correctly listed in your <b>CLASSPATH</b> environment
|
|
varaible, and your main java launcher is correctly listed in your <b>PATH</b> environment
|
|
variable.<br>
|
|
<br>
|
|
Typically, you will likely copy the jarfiles <b>aum.jar</b> and <b>xmlrpc.jar</b>
|
|
into the <b>lib/</b> subdirectory of your I2P router installation, along with all
|
|
the other I2P jar files. Wherever you choose to put these files, make sure they're
|
|
correctly listed in your <b>CLASSPATH</b>.
|
|
<br>
|
|
Also, you'll want to add execute permission to your <b>qmgr</b> (or <b>qmgr.bat</b>)
|
|
wrapper script, and copy it into one of the directories listed in your <b>PATH</b>
|
|
environment variable.<br>
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>4.3. Get Familiar With qmgr</h3>
|
|
|
|
<blockquote>
|
|
<b>qmgr</b> (or <b>qmgr.bat</b>) is a convenience wrapper script to save your
|
|
sore fingers from needless typing. It's just a wrapper which passes arguments
|
|
to the java command <b><code>java net.i2p.aum.q.QMgr</code></b><br>
|
|
<br>
|
|
You can verify you've set up qmgr correctly with the command:
|
|
<blockquote><code><pre>
|
|
qmgr help</pre></code></blockquote>
|
|
This displays a brief summary of qmgr commands. On the other hand, the command:
|
|
<blockquote><code><pre>
|
|
qmgr help verbose</pre></code></blockquote>
|
|
floods your terminal window with a detailed explanation of all the qmgr commands
|
|
and their arguments.<br>
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>4.4. Running A Q Client Node For The First Time</h3>
|
|
|
|
<blockquote>
|
|
Provided you've successfully completed the preliminaries, you can launch your
|
|
Q Client Node with the command:
|
|
<blockquote><code><pre>
|
|
qmgr start</pre></code></blockquote>
|
|
|
|
All going well, you should have a Q Client Node now running in background.
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>4.5. Verify that your Q Client Node is actually Running</h3>
|
|
|
|
<blockquote>
|
|
After typed the <b>qmgr start</b> command, you will see little or no
|
|
evidence that Q is actually running.<br>
|
|
<br>
|
|
You can test if the node is actually up by typing the command:
|
|
<blockquote><code><pre>
|
|
qmgr status</pre></code></blockquote>
|
|
If your Q Client Node is running, this <b>status</b> command should produce
|
|
something like:
|
|
<blockquote><code><pre>
|
|
Pinging node at '/home/myusername/.quartermaster_client'...
|
|
Node Ping:
|
|
status=ok
|
|
numPeers=0
|
|
dest=-3LQaE215uIYwl-DsirnzXGQBI31EZQj9u~xx45E823WqjN5i2Umi37GPFTWc8KyislDjF37J7jy5newLUp-qrDpY7BZum3bRyTXo3Udl8a3sUjuu4qR5oBEWFfoghQiqDGYDQyJV9Rtz7DEGaKHGlhtoGsAYRXGXEa8a43T2llqZx2fqaXs~836g8t6sLZjryA5A9fpq98nE5lT0hcTalPieFpluJVairZREXpUiAUmGHG7wAIjF6iszXLEHSZ8Qc622Xgwy0d1yrPojL2yhZ64o05aueYcr~xNCiFxYoHyEJO3XYmkx~q-W-mzS3nn6pRevRda74MnX1~3fFDZ0u~OG6cLZoFkWgnxrwrWGFUUVMR87Yz251xMCKJAX6zErcoGjGFpqGZsWxl4~yq7yfkjPnq3GuTxp2cB75bRAOZRIAieqBOVJDEodFYW5amCinu4AxYE7G1ezz4ghqHFe~0yaAdO74Q1XoUny138YT6P33oNOOlISO1cAAAA
|
|
uptime=4952
|
|
load=0.0
|
|
id=6LVZ9-~GgJJ52WUF1fLHt3UnH50TnXSoPQXy7WZ4GA=
|
|
numLocalItems=47
|
|
numRemoteItems=2173</pre></code></blockquote>
|
|
|
|
If you see something like this, then smile, because Q is now up on your system.<br>
|
|
<br>
|
|
If the node launch failed, you might see something like:
|
|
<blockquote><code><pre>
|
|
Pinging node at '/home/myusername/.quartermaster_client'...
|
|
java.io.IOException: Connection refused
|
|
at org.apache.xmlrpc.XmlRpcClient$Worker.execute(Unknown Source)
|
|
at org.apache.xmlrpc.XmlRpcClient.execute(Unknown Source)
|
|
at net.i2p.aum.q.QMgr.doStatus(QMgr.java:310)
|
|
at net.i2p.aum.q.QMgr.execute(QMgr.java:813)
|
|
at net.i2p.aum.q.QMgr.main(QMgr.java:869)
|
|
Failed to ping node</pre></code></blockquote>
|
|
This indicates that your Q client node has either crashed, or failed to launch in the
|
|
first place.<br>
|
|
<br>
|
|
If you're having trouble like this, you might like to try running your Q client node
|
|
in foreground, instead of spawning it off into background.<br>
|
|
<br>
|
|
The command to run a Q client node in foreground is:
|
|
<blockquote><code><pre>
|
|
qmgr foreground</pre></code></blockquote>
|
|
You should see some meaningless startup messages, and no return to your shell prompt.<br>
|
|
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>4.6. Diversion - Q Storage Directories</h3>
|
|
|
|
<blockquote>
|
|
By default, when you run a Q Client Node, it creates a datastore directory tree
|
|
at <b>~/.quartermaster_client</b>. (Windows users note - you'll find this directory
|
|
wherever your user home directory is - this depends on what version of Windows
|
|
you have installed).<br>
|
|
<br>
|
|
Within this directory tree, you should see a file called <b>node.log</b>, which
|
|
will contain various debug log messages, and can help you to rectify any problems
|
|
with your Q installation. If you hit a wall and can't rectify the problems
|
|
yourself, you should send this file to the Q author (aum).<br>
|
|
<br>
|
|
It's possible to run your Q node from another directory, by passing that directory
|
|
as a <b>-dir <path></b> argument to the
|
|
<b>qmgr</b> <b>start</b>, <b>foreground</b> and <b>stop</b>
|
|
commands. See <b>qmgr help verbose</b> for more information.
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>4.7. Importing a Noderef</h3>
|
|
|
|
<blockquote>
|
|
Note from the prior <b>qmgr status</b> command the line:
|
|
<blockquote><code><pre>
|
|
numPeers=0</pre></code></blockquote>
|
|
This means that your Q client node is running standalone, and doesn't have any contact
|
|
with any Q network. As such, your node is effectively useless. We need to hook up
|
|
your node with other nodes in the Q network.<br>
|
|
<br>
|
|
Q doesn't ship with any means for new client nodes to automatically connect to any Q
|
|
server nodes. This is deliberate.<br>
|
|
<br>
|
|
In all likelihood, there will be one 'main' Q network running within I2P, largely
|
|
based around the author's own Q server node, and most people will likely want to
|
|
use this Q network. But the author doesn't want to stop other people running their
|
|
own private Q networks, for whatever purpose has meaning for them.
|
|
|
|
<blockquote><i><small>
|
|
<hr>
|
|
This is especially relevant for Q as opposed to Freenet. With Freenet, there's
|
|
no way for a user to know of the existence of any item of content without
|
|
first being given its 'key'. However, since Q works with published catalogs,
|
|
any user can know everything that's available on a Q network, which might
|
|
not be desirable to those wishing to share content in a private situation.<br>
|
|
<Br>
|
|
The Q author anticipates, and warmly supports, people running their own
|
|
private Q networks within I2P, in addition to accessing the mainstream
|
|
'official' Q network.<br>
|
|
<br>
|
|
The way Q is designed and implemented, there is no way for anyone, including
|
|
Q's author, to know of the existence of anyone else's private Q network.
|
|
It is beyond the author's control, (and thus arguably the author's
|
|
legal responsibility), what private Q networks people set up, and what
|
|
kind of content is trafficked on these networks. This claim of plausible
|
|
deniability on the part of Q's author parallels that of a hardware retailer
|
|
denying responsibility for what people do with tools that they purchase.
|
|
<hr>
|
|
</small>
|
|
</i></blockquote>
|
|
|
|
Ok, getting back on topic - your brand new virgin Q client node is useless and lonely,
|
|
and desperately needs some Q server nodes to talk to. So let's hook up your node to
|
|
the mainstream Q network.<br>
|
|
<br>
|
|
You'll need to get one or more 'noderefs' for Q server nodes.<br>
|
|
<br>
|
|
There's nothing fancy about a Q noderef. It's just a regular I2P 'destination', with
|
|
which your Q Client Node can connect with a Q Server Node.<br>
|
|
<br>
|
|
A 'semi-official' list of noderefs for the mainstream Q network can be downloaded
|
|
from the url: <a href="http://aum.i2p/q/qnoderefs.txt">http://aum.i2p/q/qnoderefs.txt</a>.<br>
|
|
<br>
|
|
Download this file, save it as (say) <b>qnoderefs.txt</b>. (Alternatively, if you're
|
|
wanting to subscribe into a private Q network, then get a noderef for at least one
|
|
of that network's server nodes from someone on that network who trusts you).<br>
|
|
<br>
|
|
Import these noderefs into your Q client node via the command:
|
|
<blockquote><code><pre>
|
|
qmgr addref qnoderefs.txt</pre></code></blockquote>
|
|
If all goes well, you should see no output from this command, or (possibly) a brief
|
|
line or two suggesting success.<br>
|
|
<br>
|
|
Your client node is now subscribed into the Q network of your choice. Verify this
|
|
with the command:
|
|
<blockquote><code><pre>
|
|
qmgr status</pre></code></blockquote>
|
|
In the output from that command, you should see the <b>numPeers=</b> line showing at least
|
|
1 peer.<br>
|
|
<br>
|
|
If there is more than one Q Server Node on the Q network you've just subscribed to,
|
|
then your local node should sooner or later discover all these server nodes, and
|
|
the <b>numPeers</b> value should increase over time.<br>
|
|
<br>
|
|
<blockquote>
|
|
<hr>
|
|
While Q is in its early development and testing stages, the author may abdicate
|
|
the mainstream Q network, and publish nodrefs for a whole new mainstream Q network.
|
|
This will especially happen if the author makes any substantial changes to the
|
|
inter-node protocol, and/or releases incompatible new versions of Q client/server
|
|
nodes. Remember that
|
|
<a href="http://aum.i2p/q/qnoderefs.txt">http://aum.i2p/q/qnoderefs.txt</a> will
|
|
serve as the authoritative source for noderefs for the mainstream Q network within
|
|
the mainstream I2P network.
|
|
<hr>
|
|
</blockquote>
|
|
|
|
When your client node gets its noderefs to a Q network, it will periodically,
|
|
from then on, retrieve differential peer list and catalog updates from servers
|
|
it knows about.<br>
|
|
<br>
|
|
Even if you only feed your client just one ref for a single server node, it will
|
|
in time discover all other operating server nodes on that Q network, and will
|
|
build up a full local catalog of everything that's available on that Q network.<br>
|
|
<br>
|
|
Provided that your client is running ok, and has been fed with at least one
|
|
ref for a live Q network that contains content, then over time, successive:
|
|
<blockquote><code><pre>
|
|
qmgr status</pre></code></blockquote>
|
|
commands should report increasing values in the fields:
|
|
<ul>
|
|
<li><b>numPeers</b> - number of peers this client node knows about</li>
|
|
<li><b>numLocalItems</b> - number of locally stored content items, ie items
|
|
which you have either inserted to, or retrieved from, your client node</li>
|
|
<li><b>numRemoteItems</b> - number of unique data items which are available
|
|
on remote server nodes in the Q network, and which can be retrieved through
|
|
your local client node.</li>
|
|
</ul>
|
|
|
|
<blockquote>
|
|
<hr>
|
|
|
|
<h4>4.7.1. One Big Warning</h4>
|
|
|
|
If you are participating in more than one distinct Q network, then <b>do not</b>
|
|
insert noderefs for different networks into the same running instance of a
|
|
local Q client, unless you don't plan on inserting content via that client.<br>
|
|
<Br>
|
|
For instance, let's say you are participating in two different Q networks:
|
|
<ul>
|
|
<li>The 'mainstream' Q netowrk</li>
|
|
<li>A secret Q network - "My friends' teen angst diaries"</li>
|
|
</ul>
|
|
If you get a noderef for both these networks, and insert both of these into the
|
|
same running Q client node, then this local client node will be transparently
|
|
connected to both networks.<br>
|
|
<br>
|
|
If you only ever plan on retrieving content, and never inserting content, this
|
|
won't be a problem, except that you won't be able to tell which content
|
|
resides on the mainstream Q network, and which resides in the secret Q network.<br>
|
|
<Br>
|
|
The big problem arises from inserting content. Whenever you insert data through this
|
|
'contaminated'
|
|
Q client node, this node picks 3 different servers to which upload a copy of this
|
|
data. You won't have any control over whether the data gets inserted to the mainstream
|
|
Q network, or your secret Q network. You might insert something sensitive, intending it
|
|
to go only into the secret Q network, where in fact it also ends up in the mainstream
|
|
network, with consequences you might not want.
|
|
</blockquote>
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>4.8. Content Data and Metadata</h3>
|
|
|
|
<blockquote>
|
|
Whenever content gets stored on Q, it is actually stored as two separate items:
|
|
<ul>
|
|
<li>The <b>raw data</b> - whether a text file, or the raw bytes of image files,
|
|
audio files etc</li>
|
|
<li>The <b>metadata</b>, which contains human-readable and machine-readable
|
|
descriptions of the data</li>
|
|
</ul>
|
|
Metadata consists of a set of <b>category=value</b> pairs.<br>
|
|
<br>
|
|
Confused yet? Don't worry, I'm confused as well. Let's illustrate this with an
|
|
example of metadata for an MP3 audio recording:
|
|
<ul>
|
|
<li>title=Fight_Last_Thursday.mp3</li>
|
|
<li>type=audio</li>
|
|
<li>mimetype=audio/mpeg</li>
|
|
<li>abstract=upcoming single recorded in our garage last April</li>
|
|
<li>keywords=grunge,country,indie</li>
|
|
<li>artist=Ring of Fire</li>
|
|
<li>size=4379443</li>
|
|
<li>contact=ring-of-fire@mail.i2p</li>
|
|
<li>key=blah37blah24-yada23hfhyada</li>
|
|
</ul>
|
|
All metadata categories are optional. In fact, you can insert content with no metadata
|
|
at all.<br>
|
|
<br>
|
|
If you fail to provide metadata when inserting an item, a blank set of metadata will
|
|
be created with at least the following categories:
|
|
<ul>
|
|
<li><b>key</b> - the derived key, under which the item will later be retrievable
|
|
by yourself and others</li>
|
|
<li><b>title</b> - if not provided at insert time, this will be set to the key</li>
|
|
<li><b>size</b> - size of the item's raw data, in bytes</li>
|
|
</ul>
|
|
Within Q, there is a convention to supply a minimal amount of metadata. While this
|
|
is not expected or enforced, including all these categories is most strongly
|
|
recommended. These core categories are:
|
|
<ul>
|
|
<li><b>title</b> - a meaningful title for the data item, consisting only of characters
|
|
which are legal in filenames on all platforms, and which ends with a file extension.</li>
|
|
<li><b>type</b> - one of a superset of eMule classifiers, such as:
|
|
<ul>
|
|
<li><b>text</b> - plain text</li>
|
|
<li><b>html</b> - HTML content</li>
|
|
<li><b>image</b> - content is in an image format, such as .png, .jpg, .gif etc</li>
|
|
<li><b>audio</b> - content is an audio sample, such as .ogg, .mp3, .wav etc</li>
|
|
<li><b>video</b> - due to the sheer size of video files, and Q's present design,
|
|
it's unlikely people will be inserting video content anytime soon (unless it's
|
|
very short)</li>
|
|
<li><b>archive</b> - packed file collections, such as .tar.gz, .zip, .rar etc</li>
|
|
<li><b>misc</b> - content does not fit into any of the above categories</li>
|
|
</ul>
|
|
</li>
|
|
<li><b>mimetype</b> - not as important as the <b>type</b> category, but providing
|
|
this category in your metadata is still strongly encouraged. Value for this category
|
|
should be one of the standard mimetypes, eg <b>text/html</b>, <b>audio/ogg</b> etc.</li>
|
|
<li><b>abstract</b> - a short description (<255 characters), intended for human reading</li>
|
|
<li><b>keywords</b> - a comma-separated list of keywords, intended for
|
|
machine-readability, should be all lowercase, no spaces</li>
|
|
</ul>
|
|
Note that you can supply extra metadata categories in addition to the above, and that
|
|
people searching for content can search on these extra categories if they know about
|
|
them.
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>4.9. Searching For Content</h3>
|
|
|
|
<blockquote>
|
|
As mentioned earlier - in constrast with Freenet, local Q nodes build up a complete
|
|
catalog of all available content on whatever Q network they are connected to.<br>
|
|
<br>
|
|
This is a design decision, based on the choice to eliminate query traffic.<br>
|
|
<br>
|
|
The author hopes that this will result in a distributed storage network with a
|
|
high retrievability guarantee, in contrast with freenet which offers no such
|
|
guarantee.<br>
|
|
<br>
|
|
With Freenet, you only ever know of the existence of something if someone tells
|
|
you about it.<br>
|
|
<br>
|
|
But with Q, your local client node builds up a global catalog of everything that's
|
|
available within the whole network.<br>
|
|
<br>
|
|
The QMgr client has a command for searching your Q client node:
|
|
<blockquote><code><pre>
|
|
qmgr search -m category1=pattern1 category2=pattern2 ...</pre></code></blockquote>
|
|
For example:
|
|
<blockquote><code><pre>
|
|
qmgr search -m type=audio artist=Mozart keywords=symphony</pre></code></blockquote>
|
|
or:
|
|
<blockquote><code><pre>
|
|
qmgr search -m type=text title="bible|biblical|(Nag Hammadi)" keywords="apocrypha|Magdalene"</pre></code></blockquote>
|
|
As implied in the latter example, search patterns are regular expressions. This example will
|
|
locate all text items, whose <b>title</b> metadata category contains one of <b>bible</b>, <b>biblical</b> or <b>Nag Hammadi</b>, <i>and</i> whose <b>keywords</b> category contains either
|
|
or both the words <b>apocrypha</b> or <b>Magdalene</b>.<br>
|
|
<br>
|
|
Please use the search function carefully, otherwise (if and when Q usage grows) you
|
|
could be inundated with thousands or even millions of entries.<br>
|
|
<br>
|
|
If a search turns up nothing, qmgr will simply exit. But if it turns up one or more items,
|
|
it will the items out one at a time, with the key first, then each metadata entry
|
|
on an indented line following.
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>4.10. Retrieving Content</h3>
|
|
|
|
<blockquote>
|
|
Now, we're actually going to retrieve something.<br>
|
|
<br>
|
|
Presumably, after following the previous section, you will have seen one or more search
|
|
results come up, with the 'keys' under which the items can be accessed.<br>
|
|
<br>
|
|
Now, choose one of the keys, preferably for a short text item. Try either of the following
|
|
commands:
|
|
<blockquote><code><pre>
|
|
qmgr get <keystring> something.txt</pre></code></blockquote>
|
|
<i>or</i>:
|
|
<blockquote><code><pre>
|
|
qmgr get <keystring> > something.txt</pre></code></blockquote>
|
|
(both have the same effect - the first one explicitly writes to the named file, the second
|
|
one dumps the raw data to stdout, which we shell-redirect into the file.<br>
|
|
<br>
|
|
<b><i>Note - redirection of fetched data to a file via shell is not working at present. Use only
|
|
the first form till we fix the bug.</i></b>
|
|
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>4.11. Inserting Content</h3>
|
|
|
|
<blockquote>
|
|
Our last example in this walkthrough relates to inserting content.<br>
|
|
<br>
|
|
Firstly, create a small text file with 2-3 lines of text, and save it as (say)
|
|
myqinsert.txt.<br>
|
|
<br>
|
|
Now, think of some metadata to insert along with the file. Or, you can just use
|
|
the set:
|
|
<blockquote><code><pre>
|
|
type=text
|
|
keywords=test
|
|
abstract=My simple test of inserting into Q
|
|
title=myqinsert.txt</pre></code></blockquote>
|
|
|
|
Now, let's insert the file. Ensure your Q client node is running, then type:
|
|
<blockquote><code><pre>
|
|
qmgr put myqinsert.txt -m type=text keywords=test title="myqinsert.txt" \
|
|
abstract="My simple test of inserting into Q"</pre></code></blockquote>
|
|
If all went well, this command should produce half a line of gibberish, followed
|
|
immediately by your shell prompt, eg:
|
|
<blockquote><code><pre>
|
|
aRoFC~9MU~pM2C-uCTDBp5B7j79spFD8gUeu~BNkUf0=<b>$</b>
|
|
</pre></code></blockquote>
|
|
The '$' at the end is your shell prompt, and all the characters before it are the 'key'
|
|
which was derived from the content you just inserted.<br>
|
|
<br>
|
|
To avoid the hassle of copying/pasting the key, you could just add output redirection
|
|
to the above command, eg:
|
|
<blockquote><code><pre>
|
|
qmgr put myqinsert.txt -m type=text keywords=test title="myqinsert.txt" \
|
|
abstract="My simple test of inserting into Q" \
|
|
> myqinsert.key</pre></code></blockquote>
|
|
This will cause the generated key to be written safe and sound into the file
|
|
<b>myqinsert.key</b>.<br>
|
|
<br>
|
|
You can verify that this insert worked by a 'get' command, as in:
|
|
<blockquote><code><pre>
|
|
qmgr get `cat myqinsert.key` somefilename.ext</pre></code></blockquote>
|
|
(Note that this won't work on windows because the DOS shell is irredeemably brain-damaged. If
|
|
you're using Windows, you <b>will</b> have to cut/paste the key.
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>4.12. Shutting Down your Node</h3>
|
|
|
|
<blockquote>
|
|
If you've worked through to here, then congratulations! You've got your Q Client Node set up
|
|
and working, and ready to meet all your distributed file storage and retrieval needs.<br>
|
|
<br>
|
|
You can leave your client node running 24/7 if you want. In fact, we recommend you keep your
|
|
client node running as much of the time as possible, so that you get prompt catalog updates,
|
|
and can more quickly stay in touch with new content.<br>
|
|
<br>
|
|
However, if you need to shut down your node, the command for doing this is:
|
|
<blockquote><code><pre>
|
|
qmgr stop</pre></code></blockquote>
|
|
This command will take a while to complete (since the node has to wait for the I2P
|
|
java shutdown hooks to complete before it can rest in peace). But once your node is
|
|
shut down, you can start it up again at any time and pick up where you left off.
|
|
</blockquote>
|
|
|
|
<a name="server"/>
|
|
|
|
<hr>
|
|
|
|
<h2>5. Running a Q Server Node</h2>
|
|
|
|
<h3>5.1. Introduction</h3>
|
|
<blockquote>
|
|
This section describes the requirements for, and procedures involved with, running
|
|
a Q Server Node.<br>
|
|
<br>
|
|
We'll use a similar 'walkthrough' style to that which we used in the previous section
|
|
on client nodes.
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>5.2. Requirements and Choices</h3>
|
|
<blockquote>
|
|
Running a Q server is a generous thing to do, and helps substantially with making
|
|
Q work at its best for everyone. However, please do make sure you can meet some
|
|
basic requirements:
|
|
<ul>
|
|
<li>You are running a permanent (24/7) I2P Router, on a box with at least (say)
|
|
98% uptime.</li>
|
|
<li>You have a little bandwidth to spare, and don't mind the extra memory, disk and
|
|
CPU-usage footprint of running a fulltime Q server node</li>
|
|
<li>You have already been able to successfully run a Q client node.</li>
|
|
</ul>
|
|
Also, please decide whether you want your server node to contribute to the mainstream
|
|
Q network, or whether you want to create your own private Q network, or join someone
|
|
else's private network. Your contribution will be most appreciated, though, if you
|
|
can run a server within the mainstream Q network.
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>5.3. Starting Your Server Node</h3>
|
|
|
|
<blockquote>
|
|
Starting up a Q Server node is very similar to starting up a Q client node, except
|
|
that with the qmgr command line, you must put the keyword arg <b>server</b> before the
|
|
command word. So the command is:
|
|
<blockquote><code><pre>
|
|
qmgr server start</pre></code></blockquote>
|
|
Similar to Q client nodes, you can check the status of a running Q server node with
|
|
the command:
|
|
<blockquote><code><pre>
|
|
qmgr server status</pre></code></blockquote>
|
|
(Note that this command will take longer to complete than with client nodes, because
|
|
the communication passes through a multi-hop I2P tunnel, rather than just through
|
|
localhost TCP).<br>
|
|
<br>
|
|
If the status command succeeds, then you'll know your new Q Server Node is happily
|
|
running in background.
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>5.4. Joining A Q Network</h3>
|
|
|
|
<blockquote>
|
|
When a Q Server node starts up for the first time, it is in a private network
|
|
all by itself.<br>
|
|
<br>
|
|
If you want to link your server into an existing Q network, you'll have to add a
|
|
noderef for at least one other server on that network. The command to do this
|
|
is similar to that for subscribing a client node to a network:
|
|
<blockquote><code><pre>
|
|
qmgr server addref <noderef-file></pre></code></blockquote>
|
|
where <noderef-file> is a file into which you've saved the noderef for
|
|
the network you want to join.
|
|
<blockquote>
|
|
<hr><i><small>
|
|
Recall from the section on client nodes that the authoritative noderefs
|
|
for the mainstream Q network can be downloaded from
|
|
<a href="http://aum.i2p/q/qnoderefs.txt">http://aum.i2p/q/qnoderefs.txt</a>.
|
|
</small></i><hr>
|
|
</blockquote>
|
|
After you've added the noderef, subsequent <b>qmgr server status</b> commands
|
|
should show <b>numPeers</b> having a value of at least 1 (and growing, as more
|
|
server nodes come online in the mainstream Q network.)
|
|
|
|
</blockquote>
|
|
|
|
<hr>
|
|
|
|
<h3>5.5. Private Networks - Exporting Your Server's Noderef</h3>
|
|
|
|
<blockquote>
|
|
If you're planning to start your own private Q network, and want to include other
|
|
server operators in this network, then you'll have to export your server's noderef
|
|
and make it available to the others you want to invite into your network.<br>
|
|
<br>
|
|
The command to export your Q Server noderef is:
|
|
<blockquote><code><pre>
|
|
qmgr server getref <noderef-file></pre></code></blockquote>
|
|
This will extract the <i>I2P Destination</i> of your running server node, and
|
|
write it into <noderef-file>. You can then privately share this file with
|
|
others who you want to invite into your private network. Each recipient of
|
|
this file will do a <b>qmgr server addref <noderef-file></b> command
|
|
to import your ref into their servers.<br>
|
|
<br>
|
|
Don't forget that if you're running, or participating in, a private Q network, then
|
|
you'll need to run a separate client for accessing this network, separate from any
|
|
mainstream Q network client you may already be running.<br>
|
|
<br>
|
|
To start this extra client, you'll have to choose a directory where you want this
|
|
client to reside, a port number you want your client to listen on locally for
|
|
user commands, and run the command:
|
|
<blockquote><code><pre>
|
|
qmgr -dir /path/to/my/new/client -port <portnum> start</pre></code></blockquote>
|
|
You need the <b>-port <portnum></b> command, because otherwise it'll fail
|
|
to launch (if you already have a client node running off the mainstream Q network).<br>
|
|
<br>
|
|
This will create, and launch, a new instance of a Q client, accessing your private
|
|
Q network. Don't forget to import your server's noderef into this client. Also,
|
|
note that you'll have to use this same <b>-port <portnum></b> argument when
|
|
doing any operation on this client instance, such as get, put, status, search.
|
|
|
|
</blockquote>
|
|
|
|
<a name="qmgr"/>
|
|
|
|
<hr>
|
|
|
|
<h2>6. About the qmgr Utility</h2>
|
|
|
|
qmgr (or, to people fluent in Java, <b>net.i2p.aum.q.QMgr</b>), is just one simple
|
|
Q client application, that happens to be bundled in with the Q distro.<br>
|
|
<br>
|
|
It is by no means the only, or even main facility for accessing the Q network. We
|
|
anticipate that folks will write all manner of client apps, including fancy GUI
|
|
apps.<br>
|
|
<br>
|
|
Anyway, qmgr does give you a rudimentary yet workable client for basic access
|
|
to the Q network. Until fancy apps get written, qmgr will have to do.<br>
|
|
<br>
|
|
Don't forget that qmgr has very detailed inbuilt help. Run:
|
|
<blockquote><code><pre>
|
|
qmgr help</pre></code></blockquote>
|
|
for a quick help summary, or:
|
|
<blockquote><code><pre>
|
|
qmgr help verbose</pre></code></blockquote>
|
|
for the 'War and Peace' treatise.<br>
|
|
<br>
|
|
<blockquote><hr>
|
|
One crucial concept to remember with qmgr is that client and server node instances
|
|
are uniquely identified by the directories at which they reside. If you are running
|
|
multiple server and/or client instances, you can specify an instance with the
|
|
<b>-dir <dirpath></b> option - see the help for details.
|
|
<hr></blockquote>
|
|
|
|
<hr>
|
|
|
|
One last note - we strongly discourage any writing of client apps that spawn a qmgr
|
|
process, pass it arguments and parse its results. This is most definitely a path to
|
|
pain, since qmgr's shell interface is subject to radical change at any time without
|
|
notice.<br>
|
|
<br>
|
|
qmgr is for human usage, or at most, inclusion in init/at/cron scripts. Please respect
|
|
this.<br>
|
|
<br>
|
|
If you want to write higher-level clients, your best course of action is to use the
|
|
official client api library, which we anticipate will have versions available in
|
|
Java, Python, Perl and C++. If you want to write in another language, such as
|
|
OCaml, Scheme etc, then the existing api lib implementations should serve as an excellent
|
|
reference to support you in writing a native port for your own language.
|
|
|
|
<a name="contact"/>
|
|
|
|
<hr>
|
|
|
|
<h2>8. Contacting the Author</h2>
|
|
|
|
I am <b>aum</b>, and can be reached as <b>aum</b> on in-I2P IRC networks, and also
|
|
at the in-I2P email address of <b>aum@mail.i2p</b>.<br>
|
|
<br>
|
|
|
|
<hr>
|
|
|
|
<center>
|
|
Return to <a href="../index.html">Q Homepage</a><br>
|
|
<br>
|
|
<small>
|
|
<a href="#intro">Introduction</a> |
|
|
<a href="#checklist">Checklist</a> |
|
|
<a href="#serverorclient">Server?orClient?</a> |
|
|
<a href="#walkthrough">Walkthrough</a> |
|
|
<a href="#server">Server Nodes</a> |
|
|
<a href="#qmgr">About QMgr</a> |
|
|
<a href="#contact">Contact us</a>
|
|
</small>
|
|
</center>
|
|
|
|
<hr>
|
|
|
|
<!-- Created: Fri Apr 1 11:03:27 NZST 2005 -->
|
|
<!-- hhmts start -->
|
|
Last modified: Sun Apr 3 20:06:53 NZST 2005
|
|
<!-- hhmts end -->
|
|
</body>
|
|
</html>
|