Node reference¶
The node (unseenet.node.server) takes articles from clients and peers,
keeps them in its store (unseenet.core.store) and passes them on. This page
is for reading its code or writing a compatible node: what each part does,
generated from the docstrings. The rules themselves are in the UN-RFCs, above
all UN-RFC 0005 (ONNTP) and the documents each part
names.
To run a node, see Running a node.
Node: taking and keeping articles¶
Node is one class written in parts. accepting.py decides the order of the
checks; each kind of article has its own part.
The node¶
The node itself: what it is made of and how it is reached.
Belongs here: Node.__init__ (the store, the blob store, the config, the node key,
the stats), the watchers of new articles (push, UN-RFC 0008), the peers it is
configured with, what it says about itself (node_info, UN-RFC 0011), and serving
connections (serve: each connection is a Session, see connection.py).
Not here: the rules for taking an article (accepting.py and the files it points to).
Part of Node (see init.py).
NodeBase ¶
NodeBase(
store: Store,
config: NodeConfig | None = None,
key: Identity | None = None,
blobs: BlobStore | None = None,
)
watch ¶
watch(pattern: str) -> asyncio.Queue
A queue that receives the Message-ID and group of every new article whose group
matches pattern (a wildmat): push for XUN-WATCH (UN-RFC 0008).
peer_config ¶
peer_config(name: str) -> PeerConfig | None
The configured peer of that name ([[peer]] in node.toml), or None.
subscribe ¶
subscribe() -> asyncio.Event
An event that is set whenever a new article is queued for peers.
policy_fingerprint ¶
policy_fingerprint() -> str
A hash of what decides which articles from peers the node takes (#543): new groups, trusted owners, exclusions, profiles, pictures, sites, files, and what each peer may give. Postage, size and retention are left out: articles refused for those are refused for themselves.
policy_changed ¶
policy_changed() -> dict | None
At start (unseenet-node run): if the policy is not the one the node last
ran with, forget the refusals it made, reconcile and pull again, and fetch the
files of stored articles it now carries (#543). None when nothing changed (or
the first run, which only notes the policy). Otherwise what was done.
node_info ¶
node_info() -> dict
The node's information (UN-RFC 0011 section 4): name, software, node key, its
policy (new groups, postage, ripening, retention, size, blocks, profiles, pictures,
sites, files) and the operator's contact, as LIST UNSEENET-NODEINFO sends it.
serve
async
¶
serve(
host: str = "127.0.0.1", port: int = 1119
) -> asyncio.base_events.Server
Listen for ONNTP connections on host:port; each one is a Session. Tor
publishes the onion service in front of it.
The accept pipeline¶
Taking an article: the pipeline every article goes through, from a client or a peer.
Belongs here: accept and _accept - the order of the checks (structure,
duplicates, which groups a peer may send, the group and its window, postage, then
manifests, control articles or ordinary posts) and storing and forwarding what passed;
holding comments that came before their thread (UN-RFC 0030 section 4.1) and waiting
for a key-bound group's manifest (UN-RFC 0040 section 6).
Not here: the rules of each kind - postage (postage.py), manifests (manifests.py), control articles (controls.py), kept articles (kept.py), edits (edits.py), blobs (blobs.py), blocks (blocks.py).
Part of Node (see init.py).
Accepting ¶
accept ¶
accept(
raw: bytes,
source: str | None = None,
now: datetime | None = None,
from_peer: bool = False,
how: str | None = None,
staged: dict | None = None,
held: bool = False,
) -> Article
Validate and store an article. Raises ArticleError with an RFC 0005 reason.
from_peer is true for articles pushed by another node (IHAVE/TAKETHIS),
authenticated or not; source names the peer when it authenticated.
how is only for the counters: "client", "peer" or "pull". held: a
comment kept aside until its thread came (UN-RFC 0030), whose time of posting
was checked when it was posted.
Blobs¶
Blobs (UN-RFC 0027): the files articles carry, when the operator carries them.
Belongs here: a client's article needing its blobs stored or staged, the daily quotas,
storing and sweeping blobs (what nothing references, refused ones, retention), refusal
lists from configured keys, and holding an article from a peer until its blobs are
fetched (blob_wait; the fetching is node/blobfetch.py).
Not here: BLOBPOST and BLOB (blob_commands.py), the rules every node checks
(core/blobs.py).
Part of Node (see init.py).
Blobs ¶
store_blob ¶
store_blob(bid: str, octets: bytes) -> None
Keep a blob's octets on disk and note it in the store (UN-RFC 0027 section 7).
sweep_blobs ¶
sweep_blobs(retention: bool = True) -> int
Delete the blobs no stored article references, refused ones, and those past the node's blob retention unless a kept article references them (UN-RFC 0027 sections 7.1-7.3, UN-RFC 0017 section 2.1).
refusal_settings ¶
refusal_settings() -> tuple[list[str], str]
The keys whose refusal lists this node follows, and the groups they may come
from (UN-RFC 0027 section 7.3): [refusals], which covers every picture the node
carries - files, profile pictures, group emoji - whether or not it carries files
(#717); also the older [blobs] refusal_keys. None by default.
backfill_blobs ¶
backfill_blobs(limit: int = BACKFILL_MAX) -> int
Fetch the files of stored articles the node now carries but lacks (#543):
after blobs were turned on, or encrypted or groups widened. They join
the articles waiting for blobs (section 6.3), with no peer of origin, so the
follower asks every peer carrying the group; they are already stored and
forwarded, so nothing is held back. Not charged to the daily quotas (8.1):
they were posted before. Returns how many articles were added, at most
limit; the rest come with the next start or as peers send them again.
fetching ¶
fetching(bid: str) -> bool
True while the node is fetching this blob for an article it holds back
(UN-RFC 0027 section 6.3): BLOB then answers 436 (try later).
blobs_done ¶
blobs_done(message_id: str) -> None
The blobs came, or we gave up: pass the article on with the usual delay.
Session: one connection¶
Every command is a cmd_<name> method, in the part of its area.
The command loop¶
One connection to the node (ONNTP, UN-RFC 0005): the command loop and what all commands share.
Belongs here: Session.__init__, reading command lines and dispatching them to
cmd_<name> (run), a full node turning clients away (#296), CAPABILITIES, and
handing an article to Node.accept.
The commands are in the files named *_commands.py: auth, lists, reading, posting,
peering, blobs.
Part of Session (see init.py).
ConnectionBase ¶
ConnectionBase(
node: Node, reader: StreamReader, writer: StreamWriter
)
send ¶
send(line: str) -> None
Send one line to the other side (CRLF added). Buffered; run drains it.
run
async
¶
run() -> None
The command loop: greet with 200, read a command a line at a time and answer
it, until QUIT, an idle connection (IDLE_SECONDS) or the client goes.
cmd_capabilities
async
¶
cmd_capabilities(args)
CAPABILITIES (RFC 3977 5.2): what this node speaks, with the Unseenet
capabilities of UN-RFC 0005 - its size and postage limits, ripening, files and the
XUN commands.
Authenticating peers¶
Authenticating peers: AUTHINFO and node keys (XUN-PEER, UN-RFC 0016).
Part of Session (see init.py).
AuthCommands ¶
cmd_authinfo
async
¶
cmd_authinfo(args)
AUTHINFO USER/PASS (RFC 4643): a configured peer logging in with its
password (281); peers with node keys use XUN-PEER instead (UN-RFC 0016).
cmd_xun_peer
async
¶
cmd_xun_peer(args)
XUN-PEER
cmd_xun_peer_proof
async
¶
cmd_xun_peer_proof(args)
XUN-PEER-PROOF
Lists and push¶
Lists and push: LIST ACTIVE and NEWSGROUPS (ordinary groups only, UN-RFC 0040 section 3), XUN-WATCH (UN-RFC 0008), and the node's information (UN-RFC 0011).
Part of Session (see init.py).
ListsCommands ¶
cmd_list_active
async
¶
cmd_list_active(args)
LIST ACTIVE [wildmat]: the ordinary groups this node carries, with
their high and low numbers - key-bound groups are not listed (UN-RFC 0040 section 3).
cmd_list_newsgroups
async
¶
cmd_list_newsgroups(args)
LIST NEWSGROUPS [wildmat]: the ordinary groups with their
descriptions from the manifest.
cmd_xun_watch
async
¶
cmd_xun_watch(args)
UN-RFC 0008: tell the client about new articles until it sends DONE.
cmd_list_unseenet_nodeinfo
async
¶
cmd_list_unseenet_nodeinfo(args)
UN-RFC 0011: who runs this node and its policy.
Reading¶
Reading: GROUP, ARTICLE, HEAD, BODY, STAT, OVER, HDR (RFC 3977, UN-RFC 0005 section 2.4).
Part of Session (see init.py).
ReadingCommands ¶
cmd_group
async
¶
cmd_group(args)
GROUP <group> (RFC 3977 6.1.1): select a group - 211 with count, low
and high; 411 when the node does not carry it, or it was removed or dropped.
cmd_listgroup
async
¶
cmd_listgroup(args)
LISTGROUP [group [range]] (RFC 3977 6.1.2): the article numbers of the
selected group - 211.
cmd_article
async
¶
cmd_article(args)
ARTICLE <message-id>|<number> (RFC 3977 6.2.1): the whole article - 220,
or 430/423 when there is none. The node's own headers are not sent.
cmd_over
async
¶
cmd_over(args)
OVER [range|message-id] (RFC 3977 8.3, also XOVER): the overview of many
articles - 224, with the Unseenet fields of UN-RFC 0005 section 2.4.
cmd_list_overview_fmt
async
¶
cmd_list_overview_fmt(args)
LIST OVERVIEW.FMT (RFC 3977 8.4): the fields of OVER, with
Unseenet's at the end (UN-RFC 0005 section 2.4).
cmd_hdr
async
¶
cmd_hdr(args, code: int = 225)
HDR <field> [range|message-id] (RFC 3977 8.5): one header of many
articles - 225. Unseenet headers are allowed too (UN-RFC 0005 section 2.4).
cmd_list_headers
async
¶
cmd_list_headers(args)
LIST HEADERS (RFC 3977 8.6): HDR takes any header (:).
Peers and catching up¶
Peers and catching up: IHAVE, CHECK, TAKETHIS (UN-RFC 0005 section 5), NEWNEWS with the groups' state first (UN-RFC 0017), XUN-RECONCILE (UN-RFC 0040) and XUN-MANIFESTS (UN-RFC 0041).
Part of Session (see init.py).
PeeringCommands ¶
cmd_newnews
async
¶
cmd_newnews(args)
NEWNEWS <wildmat> <date> <time> [GMT] (RFC 3977 7.4): the Message-IDs of
articles that arrived since then; each group's state first - manifest, key updates and
kept articles - so a new node gets what does not expire (UN-RFC 0017).
cmd_xun_reconcile
async
¶
cmd_xun_reconcile(args)
UN-RFC 0040 section 4.3: compare key-bound state by range fingerprints.
Open to readers too, so anyone can check a node; a reader gets one exchange (MAX_ROUNDS rounds) per connection, an authenticated peer is not limited.
cmd_xun_manifests
async
¶
cmd_xun_manifests(args)
UN-RFC 0041: the current manifests of up to MANIFESTS_MAX groups at once.
One answer per line, in order, as ARTICLE/STAT answer: 220 with the article as stored (no Path/Xref), 223 if the client holds the current one, 411 if we have none, 431 for lines past _s.MANIFESTS_ANSWER_BYTES (section 2.3).
cmd_ihave
async
¶
cmd_ihave(args)
IHAVE <message-id> (RFC 3977 6.3.2, peers only): 335 send it, or 435
not wanted; then 235 taken or 437 refused.
cmd_check
async
¶
cmd_check(args)
CHECK <message-id> (RFC 4644, peers only): 238 send it, or 438
already seen.
cmd_takethis
async
¶
cmd_takethis(args)
TAKETHIS <message-id> (RFC 4644, peers only): an article streamed by a
peer - 239 taken, 439 refused, 431 try later (e.g. its blobs are coming).
Posting¶
Posting from a client: POST (with the blobs staged on this connection, UN-RFC 0027).
Part of Session (see init.py).
The store¶
Store (unseenet.core.store) is the node's SQLite database, written in parts
by area; the schema is in its __init__.py.
Articles¶
Articles: storing, finding, deleting and expiring them (UN-RFC 0012).
Belongs here: add (an article, its number in its group, and the blobs it references),
looking articles up by Message-ID or number, deleting one, what arrived since a time,
and expire, which leaves the state of every group (its manifest, live key updates
and kept articles, UN-RFC 0017 section 2) and deletes what is past the window.
Part of Store (see init.py).
Articles ¶
add ¶
add(art: Article, raw: bytes) -> int
Store an article and give it the next number in its group (created if new). Answers the number. The node has checked it; the store does not.
get ¶
get(message_id: str) -> bytes | None
The raw article with this Message-ID - or the rmgroup article of a removed group - or None.
get_meta ¶
get_meta(message_id: str) -> tuple[str, str] | None
The group and author key of an article we hold, or None.
by_number ¶
by_number(
group_id: str, number: int
) -> tuple[str, bytes] | None
The Message-ID and raw article with this number in the group, or None.
delete ¶
delete(message_id: str) -> None
Delete one article (its history entry stays, so it is not taken again).
received_since ¶
received_since(since: int) -> list[tuple[str, str]]
(Message-ID, group) of the articles that arrived since a Unix time, in
the order they came - for push and for offering to peers.
expire ¶
expire(default_days: int) -> int
Delete articles past their group's retention. Returns count.
The history¶
Opening the store, and its history: which Message-IDs the node has seen.
Belongs here: Store.__init__ (the SQLite file, the schema in __init__.py, small
migrations of older stores) and the history that makes a node take an article once
(seen, remember), and the articles refused when pulled from a peer (#132).
Part of Store (see init.py).
StoreBase ¶
StoreBase(path: str | Path = ':memory:')
seen ¶
seen(message_id: str) -> bool
True when the history has this Message-ID: taken, refused or expired.
refuse ¶
refuse(message_id: str, peer: str, reason: str) -> None
Note that an article pulled from peer was refused, so it is not fetched
from that peer again. Unlike history, this never refuses the article itself:
it may still be posted by a client, or arrive once our policy allows it.
refused_from ¶
refused_from(message_id: str, peer: str) -> bool
True when this article was refused when pulled from this peer (#132), so it is not asked for again.
remember ¶
remember(message_id: str) -> None
Remember a Message-ID in the history, so the node takes it once.
Groups¶
Groups: manifests, the groups a node carries, their state, epochs and removal.
Belongs here: the current manifest of a group (manifest, set_manifest), the
groups for LIST, a group's state for NEWNEWS (state_ids, UN-RFC 0017: manifest,
key updates, then kept articles), the epochs of closed groups (UN-RFC 0004 section 6),
and groups removed by their owner or dropped by the operator (tombstones, UN-RFC 0013).
Part of Store (see init.py).
Groups ¶
manifest ¶
manifest(group_id: str) -> Manifest | None
The current manifest; None if the group is not carried or has expired.
set_manifest ¶
set_manifest(m: Manifest) -> None
Make a manifest the group's current one; a key-bound group's is also put in the sums that reconciliation compares (UN-RFC 0040 section 4).
groups ¶
groups(
ordinary: bool = False,
) -> list[tuple[str, int, int, str]]
(group_id, high, low, description) for carried groups; with ordinary,
without key-bound groups (UN-RFC 0040 section 3: not listed).
set_epoch ¶
set_epoch(
group_id: str,
epoch: int,
member_key: bytes,
message_id: str | None = None,
) -> None
Record a closed group's key epoch and its member key, with the key update that made it (UN-RFC 0004 section 6).
state_ids ¶
state_ids(group_id: str) -> list[str]
UN-RFC 0017: the articles a node needs before it can accept posts in a group - its current manifest and the key updates of the live epochs.
removals_since ¶
removals_since(since: int) -> list[tuple[str, str]]
(rmgroup Message-ID, group) for groups their owner removed since since.
close_previous ¶
close_previous(group_id: str, epoch: int) -> None
Note that a closed group's epoch has been replaced: posts encrypted to it
are no longer taken (UN-RFC 0004 section 6).
closed_before ¶
closed_before(group_id: str) -> int
Articles of epochs below this are refused (0: none closed).
epochs ¶
epochs(group_id: str) -> dict[int, bytes]
A closed group's epochs and the member key of each (UN-RFC 0004 section 6).
tombstone ¶
tombstone(group_id: str) -> str | None
Why a group is gone - owner (removed, UN-RFC 0013) or operator
(dropped) - or None.
add_tombstone ¶
add_tombstone(
group_id: str,
reason: str,
message_id: str | None = None,
raw: bytes | None = None,
) -> None
Mark a group removed - by its owner (owner, with the rmgroup article,
UN-RFC 0013) or dropped by the operator (operator) - so it is not taken again.
dropped_groups ¶
dropped_groups() -> list[str]
Groups the operator has dropped (UN-RFC 0013 section 3), sorted.
lift_tombstone ¶
lift_tombstone(group_id: str) -> bool
Take back an operator's drop of a group, so it can be carried again; an owner's removal stays. True if there was one.
remove_group ¶
remove_group(group_id: str) -> int
Delete a group's manifest and articles. Returns the number of articles deleted.
Kept articles¶
Kept articles (UN-RFC 0017 section 2.1): what a group keeps beyond its window.
Belongs here: working out the kept articles of a group from what the node holds and
the current manifest (kept_ids): pins and unpins (UN-RFC 0033: markers,
pins_in_force, a pin following an edit) and the pages of sites (UN-RFC 0035:
page_versions, site, site_kept - current and previous versions). A new kind
of kept article adds its table to the schema and its part to kept_ids.
Part of Store (see init.py).
Kept ¶
add_marker ¶
add_marker(
art: Article, action: str, target: str, replaces=()
) -> None
Remember a pin or unpin of target (UN-RFC 0033), with the markers it
replaces (Supersedes).
markers ¶
markers(
group_id: str, actions=(pn.PIN, pn.UNPIN)
) -> list[pn.Marker]
Every marker held for a group of the given actions: pins and unpins
(UN-RFC 0033) by default; comments-close/comments-open (UN-RFC 0030 3.2)
live in the same table.
marker_action ¶
marker_action(message_id: str) -> str | None
The action of a marker we hold (pin, unpin, comments-close, comments-open).
comments_closed ¶
comments_closed(
group_id: str,
target: str,
target_author: str,
manifest: Manifest | None = None,
) -> pn.Marker | None
The close that keeps comments on target closed, or None (section 3.2).
root_days ¶
root_days(root: str) -> int | None
The shortest Comment-Days an edit of the thread root root gave it.
set_root_days ¶
set_root_days(root: str, group_id: str, days: int) -> None
An edit of a root shortened its time for comments (section 3.1): kept shortest.
marker ¶
marker(message_id: str) -> tuple[str, str] | None
(group, target) of a pin or unpin we hold.
current_version ¶
current_version(message_id: str) -> str
An edited post's current version (UN-RFC 0042); the post itself otherwise.
withdrawn ¶
withdrawn(group_id: str) -> set[str]
Targets withdrawn by a valid cancel (UN-RFC 0033 section 4.2, step 2).
pins_in_force ¶
pins_in_force(
group_id: str,
manifest: Manifest | None = None,
extra: list[Marker] = (),
) -> list[pn.Marker]
The pins in force (UN-RFC 0033 section 4.2), newest first, also those whose
target has not arrived yet (pending). extra: markers not stored yet.
kept_ids ¶
kept_ids(
group_id: str,
*,
window_days: int | None = None,
now: float | None = None,
) -> list[str]
The group's kept articles in listing order (UN-RFC 0017 sections 2.1, 3): pins in force, the latest unpin of a target while inside the window, then the pinned targets. At most 256 targets (section 2.2).
add_page ¶
add_page(
art: Article, path: str, removed: bool, supersedes=()
) -> None
Remember a page article of a site: its path, whether it is a removal, and the versions it replaces (UN-RFC 0035).
page_versions ¶
page_versions(group_id: str) -> list[pg.Version]
The page versions of a site that count: held, and not cancelled (UN-RFC 0035 section 5.1).
site ¶
site(
group_id: str,
manifest: Manifest | None = None,
extra: list[Version] = (),
) -> tuple[dict, list[str]]
(current versions by path, paths in force) of a site (sections 5.1, 5.2).
site_kept ¶
site_kept(
group_id: str,
manifest: Manifest | None = None,
since: float = 0,
extra: list[Version] = (),
) -> list[str]
Section 6.1: the current and previous version of every page in force, by path, and a current removal while inside the window.
Blobs in the store¶
Blobs (UN-RFC 0027): which articles reference which blobs, and the operator's limits.
Belongs here: the blobs a node stores and since when, the references from articles,
what may be deleted (nothing references it, refused, past retention unless kept),
refused blobs (section 7.3), and the day's octets per key and group (section 8.1).
The octets themselves are files (core.blobs.BlobStore).
Part of Store (see init.py).
Blobs ¶
note_blob ¶
note_blob(bid: str, size: int) -> None
Note that the node stores a blob, and since when (for retention).
forget_blob ¶
forget_blob(bid: str) -> None
Forget that the node stores a blob (after its file was deleted).
articles_missing_blobs ¶
articles_missing_blobs() -> list[
tuple[str, str, list[tuple[str, int]]]
]
Stored articles that reference blobs this node neither stores nor refused: (Message-ID, group, [(blob ID, octets)]), for fetching them once the node carries them (#543).
blob_refs ¶
blob_refs(message_id: str) -> list[tuple[str, int]]
The blobs an article references: (blob id, size).
blobs_to_delete ¶
blobs_to_delete(
retention_days: int | None, now: float | None = None
) -> list[str]
Section 7.1: stored blobs no stored article references, refused ones, and - beyond the node's blob retention - those no kept article references.
kept_blobs ¶
kept_blobs(group_id: str | None = None) -> dict[str, int]
Blobs referenced by kept articles (UN-RFC 0017 2.1), {blob ID: octets}.
blob_stats ¶
blob_stats() -> dict
For the monitor: stored blobs, their octets, refused ones, today's use.
refuse_blob ¶
refuse_blob(
bid: str, reason: str = "", source: str = "operator"
) -> None
Refuse a blob: it is deleted and never taken again (UN-RFC 0027 section 7.3).
source is who refused it: the operator, or a key whose refusal list the node
follows.
blob_refused ¶
blob_refused(bid: str) -> bool
True when this node refused the blob (UN-RFC 0027 section 7.3).
blob_usage ¶
blob_usage(
kind: str, who: str, day: str | None = None
) -> int
The octets of blobs taken today (UTC) from one key or for one group
(kind key or group), for the daily quotas (UN-RFC 0027 section 8.1).
charge_blobs ¶
charge_blobs(key: str, group: str, octets: int) -> None
Count the octets of an article's new blobs against today's quotas of its author and its group; counts older than three days are dropped.
Cancels and edits¶
Withdrawn articles: cancels (UN-RFC 0004 section 7) and edits (UN-RFC 0042).
Belongs here: remembering cancels (also of articles still to come, and of targets we held, so a pin of them stops counting) and which version replaced which, with the date of the earliest version for comments in channels (0042 section 3.2).
Part of Store (see init.py).
Withdrawals ¶
add_cancel ¶
add_cancel(
target: str, author_key: str, group_id: str
) -> None
Remember a cancel of target by author_key (UN-RFC 0004 section 7), also
when the target has not arrived yet, so it is refused when it does.
cancel_for ¶
cancel_for(target: str) -> tuple[str, str] | None
The cancel remembered for target: (author key, group), or None.
add_supersede ¶
add_supersede(
old: str,
new: str,
author_key: str,
group_id: str,
kind: str,
first_date: int | None,
) -> None
Record that new replaces old - held and withdrawn, or not seen yet
(a pending supersede, section 3.1). The newest edit wins.
supersede_for ¶
supersede_for(old: str) -> tuple[str, str, str, str] | None
(edit, author_key, group_id, kind) for a replaced version, or None.
first_date ¶
first_date(message_id: str) -> int | None
The Date of the earliest version known of a post, given any version of it (section 3.2); None if it was never edited or no date is known.
date_of ¶
date_of(message_id: str) -> int | None
The Date of an article we hold, as a Unix time, or None.
Peer queues and pull state¶
Peers: what is queued for each, and how far the node has pulled from each.
Belongs here: the peer queues (enqueue, due, reschedule - also when blobs
came, UN-RFC 0027 section 6.3), the pull state (UN-RFC 0017), when to reconcile next
(UN-RFC 0040 section 6), and the policy the node last ran with, so a changed policy
forgets the refusals it made (#543).
Part of Store (see init.py).
Peers ¶
pulled_since ¶
pulled_since(peer: str) -> int | None
How far the node has pulled from a peer (a Unix time for NEWNEWS), or None.
set_pulled ¶
set_pulled(peer: str, since: int) -> None
Record how far the node has pulled from a peer.
reconcile_due ¶
reconcile_due(peer: str, now: float | None = None) -> bool
Never reconciled with peer, or its next time has come (UN-RFC 0040, 6).
set_reconciled ¶
set_reconciled(peer: str, next_at: float) -> None
When to reconcile key-bound state with a peer next (UN-RFC 0040 section 6).
policy_seen ¶
policy_seen() -> str | None
The fingerprint of the policy the node last ran with, or None (never, or a store from before #543).
set_policy ¶
set_policy(fingerprint: str) -> None
Note the policy the node runs with now (see policy_seen).
forget_policy_refusals ¶
forget_policy_refusals() -> int
After a change of policy: forget the articles refused for it (reason
group: not carried, sites, profiles, pictures, new groups, a peer's accept,
132), reconcile with every peer at once (UN-RFC 0040 section 6), and pull from¶
each again from the start of retention (UN-RFC 0017) - what the node already has is not fetched again, but what it refused is listed once more. Refusals for the article itself (postage, signature, permission) stay. Returns how many refusals were forgotten.
enqueue ¶
enqueue(
peer: str, message_id: str, not_before: float
) -> None
Queue an article for a peer, to be sent from not_before on (the random
forwarding delay, UN-RFC 0005 section 5).
due ¶
due(peer: str, limit: int = 100) -> list[str]
The Message-IDs queued for a peer whose time has come, oldest first, at most
limit.
dequeue ¶
dequeue(peer: str, message_id: str) -> None
Take an article off a peer's queue: sent, or not wanted.
reschedule ¶
reschedule(message_id: str, when) -> None
Move an article queued for peers to when(peer) (unix seconds), for
example when the blobs it waited for have come (UN-RFC 0027 section 6.3).
postpone ¶
postpone(
peer: str, message_id: str, seconds: float
) -> None
Try an article for a peer again in seconds (e.g. it answered 431).
Key-bound groups¶
Key-bound groups (UN-RFC 0029, 0040): their lifetime, and keeping them in step.
Belongs here: when a key-bound group's manifest expires and removing it after, and the
running sums by key prefix that reconciliation compares between peers (UN-RFC 0040
section 4: kb_items, kb_count_sum).
Part of Store (see init.py).
KeyBound ¶
expired ¶
expired(group_id: str, now: float | None = None) -> bool
A key-bound group whose manifest has run out: not carried any more (section 5).
known_group ¶
known_group(group_id: str) -> bool
The node has a manifest for the group, current or expired.
remove_expired_key_bound ¶
remove_expired_key_bound(
now: float | None = None,
) -> list[str]
Remove key-bound groups expired more than REMOVE_AFTER ago (section 5).
kb_sweep ¶
kb_sweep(now: float | None = None) -> int
Take groups whose manifest has expired out of the state (UN-RFC 0029 section 5: an expired group is absent). A renewal puts them back.
kb_rebuild ¶
kb_rebuild() -> None
Build the index from the groups table: for stores from before it existed.
kb_items ¶
kb_items(
lo: int = 0, hi: int = FULL
) -> list[tuple[bytes, str]]
(key, Message-ID) of the items with lo <= key < hi, by key.
kb_count_sum ¶
kb_count_sum(
lo: int = 0, hi: int = FULL
) -> tuple[int, int]
Count and sum of the items with lo <= key < hi.
Whole prefixes come from the running sums; only the parts of the range finer than KB_LEVELS nibbles at its two edges are read item by item.
Numbers for the operator¶
Numbers for the operator's monitor (node/stats.py): groups, articles, bytes, queues.
Part of Store (see init.py).