chromosomix 0.12.0
What changed in chromosomix 0.12.0, released on 10 September 2026.
-
Find a sequence — a new analysis task. Choose a loaded reference, type a motif, and every place it occurs comes back as two lanes. The pattern is IUPAC —
GGTCANNNTGACC,GATC,TTAGGG— spaces are ignored, and both strands are searched unless you say otherwise; a palindrome is reported once rather than twice, because searching both strands ofGAATTCwould otherwise count every site again and a reader totting up a lane would get double the answer with nothing on screen to explain it.Two lanes, not one, and that is the point. A set of sites answers two questions and no single picture answers both: where exactly is it, which is an interval per site — clickable, strand-carrying, exportable as BED, and an eligible input to overlap, the metaplot and APA like any other interval set (§18.3) — and how dense is it here, which is sites per bin and is the only one of the two that means anything at chromosome scale. The first is a picket fence when you zoom out; the second is a bar chart of ones when you zoom in.
Two decisions inside it are worth stating because they are the ones that would be wrong by default. An
Nin the reference does not satisfy a pattern that names a base: an assembly gap is a base nobody knows, not a base that is any of four, and matching it would turn every gap into a solid bar of fictional hits — most of what you would see on a draft genome. And an empty bin in the density lane is a zero, never a gap:NaNin a per-bin vector draws as a hole, and “there is no site here” is the answer, not missing data.It is a registered task rather than a card of its own, which is what earns it the parameter form, a recipe that travels in the session, provenance on both outputs, a place in the deck’s all-maps overlay, and Recompute on a reload — none of which a bespoke card would have had, and all of which every other task already has.
venue: 'annotations', because the residues are in that worker and nowhere else: a genome is parsed into it and deliberately never shipped out whole, so searching anywhere else would mean hauling megabytes across the boundary or stitching thirty windowed reads together at the seams. A run declines a pattern that is not IUPAC before it starts, with the character it could not read named.It works in a genome-browser session as well as over a map, which is the session a reader who came in through a FASTA is actually in. That took fixing something older: a session with no contact map reported its bin size as
0, a placeholder that was safe only while nothing producing a per-bin result could be run without a matrix. A task answering from the sequence broke that assumption, and a zero bin size gives every contig zero bins — a lane that is permanently empty rather than one that is visibly wrong, which is the failure nothing reports. A mapless session is now given a browsing grid chosen from the assembly’s own length; it is a resolution to draw at and never a claim that anything was measured at it, so the tasks that genuinely need a matrix go on saying they are waiting for one. -
A fold button on every panel header. Each of the five columns now carries a chevron beside its shortcut chip, pointing the way that column travels — left for the three on the left, right for the two on the right. The affordance existed when a panel was folded (the rotated spine tab, which points the other way because it offers to open) and was missing when it was open, so the only way to close a column was a button in the header bar that does not look like it belongs to it. It goes through the same
foldevery other entry point does, so the remembered state, the relayout and the settle all happen once. -
Shift-click ticks a range in the Tracks card. Ticking the first ten of thirty tracks was ten clicks on an eleven-pixel box, which is the arithmetic that makes a batch action cost more than doing the ten by hand. Click one box, shift-click another, and every row between the two takes the state the clicked box is moving to — the gesture every file list has. The anchor is remembered as a row’s id and resolved to a position at click time, because the rows are drag-reorderable and the card’s list is not the store’s: an index cached at the first click is a lie the moment anything moves.
-
Z-score the tracks first, on the metaplot. A new option under Summary, off by default. Two tracks in different units — a CPM bigWig and a log2 ratio — plotted on one axis compare nothing; standardising each by its own mean and SD before the metric puts them on one scale. The statistics are taken per track and shared across that track’s ranking groups, which is the part that is easy to get wrong: standardising each group separately would re-centre every one of them on zero and erase the ranking the reader split them by. A constant track is centred and not divided, so nothing comes back as ±∞, and gaps stay gaps. The heatmap shows the standardised values too, so the two halves of one figure cannot disagree, and the y axis is relabelled rather than left claiming a unit the numbers no longer have.
-
The viewer is published to npm, as a built directory rather than as something to import — so a page can embed a copy that depends on nobody’s hosting, including ours:
<iframe src="https://cdn.jsdelivr.net/npm/metacooler-viewer@0.11.0/dist/index.html?embed=1&config=…"></iframe>Terminal window npm pack metacooler-viewer && tar xf metacooler-viewer-*.tgz && cp -r package/dist /var/www/viewer/A sixth invocation of the same bundler (
vite.package.config.ts), and the one built for a path it does not know. Everything else this repository emits is served from somewhere decided in advance — the site root,/v/<commit>/, an asar, a disk — and bakes absolute asset URLs accordingly. A tarball is unpacked wherever somebody puts it, so its base is./and one published version serves all of them. The rule that comes with that is the mounter’s: the URL has to end in a slash, or nameindex.html.Two things follow from a copy not knowing where it is, and both are enforced rather than assumed (
IS_PORTABLE): it registers no service worker, since a copy mounted under a path it does not own must not claim a scope there — on a registry CDN that scope covers other people’s packages — and it asks for no version manifest, sinceversions.jsonlives at a site root a vendored copy has no way to find. Before those guards, every vendored copy would have made two doomed requests: a 404 for asw.jsthis build does not emit, and a cross-origin fetch for a manifest belonging to somebody else’s server.filesisdist/anddist-cli/, which is the whole of what stands between a publish and 250 MB of demo data, and which makesnpx metacooler-runwork against a published version with no clone.prepackbuilds andcheck-package-build.mjsthen refuses the result if it is wrong — a tarball packed over a site build would carry absolute asset URLs that 404 on every mount but the site root, and unlike everything else here a published version cannot be corrected.private: trueis gone with its comment. The argument it made — an application, not a library — is still true and is not what publishing changes: what a registry is wanted for here is hosting nobody has to trust, not dependency resolution. Specified at SPECS §22. -
A
LICENSEfile, with the MIT text the manifest has always declared. It was missing, which matters most at the moment a package is published: npm shows the licence of a package it can find one for, and a tarball that declares MIT without carrying it is asking to be taken on trust. -
.pairsfiles open, at a resolution you choose. A 4DN pairs file — plain, gzipped or bgzipped, with a.px2beside it or without one — is now an accepted input, and it is the first one the app imports rather than opens. Everything else in §3 is a matrix in a container; a pairs file is a list of contacts, and there is no grid in it. So the first open is refused, a dialog asks for a base resolution and a number of ladder rungs, and the file is read once. After that it is a map like any other: the same views, the same deck, the same turn, the same fourteen analysis tasks, none of which learns the container exists.The grid could not be guessed, and that is why there is a dialog rather than a default. A bin table inferred from the largest coordinate observed would differ between two subsamples of the same library — the one way of being wrong that survives every check downstream and makes two files silently incomparable.
The ladder is nearly free, so it is offered rather than rationed: the base rung costs a pass over the records, and every rung above it is derived from that table, which is smaller than the record stream by the ratio of records to stored pixels — thirteen to one on
demo/yeast/. The rungs are spelled the way an.mcool’s resolutions are, which wires them into the resolution picker, the refine ladder and a deck descending to its lead’s rung with no code in any of them.The shape of the data is what makes this the right design. On this repository’s own demo files, yeast Micro-C is 179 M records and 13.7 M stored pixels at 1 kb; the mock community is 10.9 M records and 7.1 M pixels. The input is gigabytes and the output is tens of megabytes — so the file is paid for once, at open, and every view afterwards costs no I/O at all.
demo/ecoli/SRR19500734.pairs.gz— 238 MB, 17.7 M records — imports in about forty seconds.What decides that time is the file’s BGZF framing rather than this code, and it is worth knowing because it can be changed. A deflate block costs about the same to start whether it holds 1 KB or 64 KB, so a finely framed file pays that fixed cost tens of thousands of times over: measured on the same 1.39 M records, 0.69 M records/s at the 64 KiB blocks
bgzipwrites and 0.12 M records/s at the ~1.4 KB blocks that E. coli file has. Re-blocking it (bgzip -d < f.pairs.gz | bgzip > g.pairs.gz) is worth about 5×.An imported map is a fuller citizen than a
.hic: because the file was read, the per-bin pixel index is a by-product rather than a whole-file scan, sobinOffsets()answers where a.hic’s refuses andnnzis a real number.What it is not is a
.metacool.#chromsize:has no concept of a genome, so an imported map is a cooler-class citizen (§9) — chromosomes table, no genomes table, the four metagenome tasks absent rather than disabled. -
Switching resolution on an imported map no longer re-reads the file. The rungs are spelled as an
.mcool’s resolutions — which is what wires them into the picker, the hash and the refine ladder for free — so a switch arrives as an ordinary open carrying no binning, and the ordinary path would drop the ladder and ask for a bin size that had already been given. -
metacooler-runtakes a.pairstoo, with--bin-sizeand--rungsstanding in for the dialog the browser shows. Everything after the open is the same code, which is the whole point of the CLI: a number computed in a batch and a number computed by clicking come out of one implementation. Note the older--pairsflag is a BEDPE of features for a pileup — a contact list is the positional argument, and the usage now says so. -
The
.mcoolis written back out, and downloads when it is ready. Every rung, as a real cooler:chroms,bins,pixels,indexes, cooler’s own format attributes, and the pairs preamble carried verbatim inmetadatabeside the record count and whatever the import skipped, clamped or had to merge. Raw counts and no weights —generated-bysaysunbalanced, andcooler balanceis the answer. It runs beside the open map rather than in front of it, because the reader has already waited once and what they are waiting for now is a file rather than a picture.This required narrowing a stated non-feature, and it is narrowed in the open at SPECS §18.9 rather than stepped around. Writing a matrix derived from an opened map is still refused, for the reason that clause gave: provenance, balancing and the bin table would be choices this app makes on the reader’s behalf and then seals into a file that outlives the session. A file binned from a
.pairsanswers all three from its input — the bin table from#chromsize:and a size the reader picked, provenance from the header, and no balancing at all — and what is written is what is on screen, off the arrays the map is drawn from, so there is no second computation to disagree with the first.Checked two ways, because a round trip alone would only prove the reader and the writer agree — which is exactly what they did about
.hicendianness: the file is read back through this app’s own reader and compared cell for cell against an independent binning of the same records written the slow obvious way. One structural difference fromcooler cloadis stated rather than left to be found:bins/chromis int32 codes rather than an HDF5 enum, because h5wasm exposes no enum construction. Everything numeric is identical. -
The viewer, inside somebody else’s page.
?embed=1mounts the same web build with no window around it — no header, no tab strip, no status bar — so it can be the figure in a portal, a lab site, a course page or a paper’s supplementary material. It is an<iframe>and a URL: no backend, no package to install, and no header for the host page to set.That last one is why this is possible at all. Cross-origin isolation (
COOP/COEP) has to be arranged by the hosting page and almost never is, which is the usual reason a wasm application cannot be embedded. This viewer has never needed it — h5wasm 0.10.x is single-threaded with the binary inlined — so the two prerequisites that are hard were already met: every view state is in the hash, and every push is archived at…/v/<commit>/, so an embed in a 2026 paper still renders in 2031.Three ways to say what to show, in the order they override one another:
#s=(a whole session deflated into the link, what Share already produced),#uri=…(one file at one view, what a navigation leaves behind), and the new?config=<url>— a published.metacooler.jsonnamed in the address, fetched, signed where a bucket entry covers it, and resolved against its own URL, through the roadSession.openConfigUrlalready took for the picker’s URL box.?config=is the pointer form of what#s=carries as a payload: editable after the embed is published, of no length, and self-contained enough that the folder can be copied to another bucket.And a
postMessagechannel, so an embed is not only a picture:readyandstateout,whereandnavigatein. Anavigateis a patch over what is on screen —{ locus: … }moves the browser without the page knowing what else is in the state — and it is serialised and parsed back through §5.5’s own reader rather than believed, because an object from another origin is exactly as untrusted as a hand-edited link. The frame answers only the window that framed it, only after that window has spoken, and only at the origin it spoke from: a reader can open their own file in an embed, and the URL that follows is not the host’s to learn unless it asked to be told.One bar, and two of the five columns. Chrome-less is not featureless: the bar carries the mark and the name — a link out to this same view in the full application, in a new tab — the file’s own breadcrumb, and on the right the Annotations and Aesthetics toggles beside About, which is where a figure in somebody else’s page says what it is, that nothing is uploaded, and which build drew it. Those two columns stay because they are controls over the picture; Files, General and Analysis are about the window. They arrive folded, standing as named spines on either edge — a column is open until somebody folds it, and a frame that honoured that came up with five open columns and thirty pixels of map between them — and they neither read nor write the fold and width kept for the full app, so a reader who opens a panel inside an embedded figure does not find the application rearranged the next time they go to it.
Specified at SPECS §21, with a picture of a real one and the worked example in the README.
-
A progress bar on the two opens that are worth watching, with a way out. Almost every read here is bounded and finishes before anyone could want news of it. Two are not: a HiC-Pro pair is read whole and parsed at open (§3.9), and an
.scoolover the network pays its listing cost up front (§3.8) — and a spinner in front of either gives a reader no way to tell a slow connection from a hung one, and no way to stop.So the loading screen now talks back. It stays a spinner until the worker has something to say, and from the first word it is a labelled bar and a cancel: bytes while a file is coming across, pixels while it is parsed — two sweeps with a denominator each, rather than one bar sliding between two and appearing to go backwards. Cancelling is not a failure and does not look like one: it stops the download mid-stream and goes back to the picker, the way dismissing the gate does. A 258 MB pair reads in about four seconds, and the bar spends three of them on the download.
The
.scoollisting is the one wait that cannot be measured while it happens — it is a single call into HDF5 that issues a thousand small reads and returns when it has them, so the thread is not free to say anything until it already has the answer. What it can say is which wait it is, before it starts: an indeterminate bar under the name of the phase. -
A cell sum says how far it has got. Summing a cell set is the one read in this application that is linear in something other than the picture: the same window that draws instantly over one cell is four thousand small reads over a whole file, and the veil in front of the map said
loading…for all of it. It now counts cells —summing 1,204 of 3,882 cells…— in every map view, and the whole-file view’s own bar names the unit it is counting rather than calling three hundred cells three hundred pixels. -
HiC-Pro’s
.matrix+_abs.bedpair, as one contact map. The output of the pipeline most published Hi-C was processed with, and it opens as what it is: the BED is the coordinate system, the matrix is the pixels, and between them they are an ordinary map — the chromosome table, every region, the analysis panel, tracks, export, the deck. All fourMATRIX_FORMATvariants read,iced/matrices included.The bin file is found three ways, because it is not optional. A
.tbiis an optimisation and a.baiis a format’s only bounded read; an_abs.bedis the matrix’s coordinate system, so without it there is not a slower map but no map. So it is paired when both are dropped together, handed over when the matrix is picked out of a connected folder, and probed for beside a URL where HiC-Pro writes it —raw/andiced/share one bed per resolution, so the_icedmatrices are tried against the raw directory’s too. A URL whose bin file is somewhere unguessable can name it:<matrix>#bins=<url>. A matrix with none of the three is refused by name, and the refusal lists all three.Read whole once, and then it is a cooler. This is the one format here that cannot be indexed: the rows are text of variable width, so byte offset and bin id have no relation, and
asissays outright they need not be sorted. So the matrix is parsed at open — byte by byte, never a string per line, at a measured 6.8 M pixels/s — and a counting sort bybin1produces the sorted triplets and the cumulative index together. After that every read is bounded and in memory, and panning costs nothing.The sort behind that is a radix sort — two stable counting passes, by column then by row — and not a sort per row band, which is what it was first. Bands are wide: a row of a 100 kb human matrix has thousands of non-zero columns, and sorting each on its own is quadratic in that width. On a 211 MB file it took 59 seconds where the radix sort takes 3.7, and the parse itself runs at 100 MB/s — the sort must not be what decides the wait. The second pass’s bucket boundaries are the cumulative
bin1_offset, so the index falls out of the sort rather than being built after it.Which is why there is no
HicProStore. A.hiccannot answerpixels/*orindexes/bin1_offsetand needs a pixel store of its own; this container was read whole by necessity, so it serves those datasets honestly andCoolerStorereads them unchanged. The pixel path for a HiC-Pro map is the cooler path, byte for byte.A mirror and a repeat are not the same duplicate. Two rows can land on one cell and they mean opposite things:
(a, b)and(b, a)is one contact stated twice, whichMATRIX_FORMAT=completewrites and which must be dropped — adding it doubles every off-diagonal cell and leaves the diagonal alone, giving a picture that still looks entirely like a contact map. Two rows on the same side are more evidence, and are summed. Once both have been folded onto the upper triangle they are indistinguishable, so the side each row was stored on is carried as far as the collapse.Not read, and both deliberate: fragment-level maps (
GENOME_FRAGMENT), since a fragment grid is variable bins under another name — refused with the config key that produced it; and the.biasesfile, since those biases normalise the raw matrix while theiced/one is already normalised, and offering them as a weight column would let somebody normalise twice. -
A second load gate, on bytes rather than on the network. Past 200 MB of matrix the reader is told what the read costs — the size, the pixel count it works out to, and that it is paid once rather than per view, which is the part that turns a bad number into an acceptable one. It shares a module with the
.scoolgate below because the two are the same shape; they differ in that an.scooloffers two readings and this offers one, and that an.scool’s cost is round trips so a local file is never asked, where this one’s is parsing and memory and a local file pays in full. The panel also names the way out for a map too fine to hold:cooler load -f cooturns the same two files into a.coolthat opens in a handful of requests at any size. -
.scool— single-cell coolers, per cell and summed. The container from Wolff et al. 2021: one cooler per cell over one shared bin space. It opens as what it is. A Cells table is the landing view — every cell in the file, with thennzand contact total its own attributes record, sortable and filterable like the genomes table beside it — a click opens one cell’s map, and ticking any number of them adds them together. Tick all of them and that is the pseudo-bulk.Both layouts, and neither is the special case. The standard
/cellscontainercooler.create_scoolwrites, and the flat one HiCMatrix and scHiCExplorer produced before the format was standardised in cooler 0.8.11 — cells at the file root, no shared tables, and a root that announces nothing. The second is the majority of what exists (every published.scoolpredating 2021 is one), so it is detected by elimination and tested at every step alongside the first.Summing needs no arithmetic, and that is the whole design. Every cell indexes the same bins, so a bin id means the same thing in all of them; and the dense scatter was already additive per pixel, because a bounded rectangle may arrive in several chunks. A store that hands one callback each selected cell’s chunks in turn therefore is their sum — no summing code, no intermediate matrix, a peak footprint of one chunk. The same falls out twice more:
nnzis a total over the set, and a sum of cumulative indexes is the cumulative index of the sum, so the density proxy, the band planner and the selection read all work over a cell set without knowing one exists.The set is a property of the file, not of the view. Chosen cells stay chosen through a chromosome and back out, they show in the breadcrumb in every view — a map summed from one cell and one summed from four thousand are different experiments and that must not be inferrable only from the picture — and they travel in the link as
cells=, with*for all of them, because four thousand names is not a URL. Balancing sums the raw counts and applies the file’s shared/bins/weightonce to the result, which is cooler’s own definition applied to the pseudo-bulk; it is offered only where such a column exists. One cell is addressable on its own asfile.scool::/cells/<name>and is an ordinary cooler there, weights included. -
A gate before a remote
.scoolis read, with both costs on the buttons. The first and only open in this viewer that asks a question, and the reason is worth stating because it is a property of HDF5 rather than of the format: a container’s cell names live in the group’s link storage, which the library allocates as members are added — immediately after each cell’s pixels, for every writer there is. The index of the file is interleaved with its data rather than gathered anywhere.Measured on the 3,882-cell Nagano 2017 file: listing the cells is 1,015 reads averaging 512 bytes, spread uniformly across all 327 MB. Not a quirk of that writer — a container written by
cooler.create_scoolbehaves identically, and a synthetic 800-cell 50 MB one touches all 48 of its megabyte blocks. So no chunk size wins: the reads are ~500 bytes about 320 KB apart, so anything large enough to coalesce them fetches the whole file and anything small enough to avoid that needs a round trip each. Rather than pick one silently, the reader is offered both with the numbers on them — one download, or ~1,000 small requests for ~5% of the bytes — and told that a local copy costs neither. A local file is never asked: the question is about round trips, and a file on disk seeks for free.The consolation is that the listing is the only cost. Every cell’s attributes sit in metadata the listing already brought across — measured at zero additional chunks on both files — so the cells table’s columns are free once the names are paid for.
-
A despeckle whose window sizes itself, in the
smoothpicker. One new setting,adaptive, beside the fixed3 × 3,5 × 5and7 × 7. Each cell’s window grows until it covers enough evidence to average, so it stays narrow where the counts are thick and widens where they thin out. Carried in the link assmooth=a50, and written into the colourbar caption ascounts · smoothed adaptively.Why a fixed window was never going to work away from the diagonal. The precision of a box mean depends on the counts it covers, not on the cells it covers. Near the diagonal a
7 × 7box holds thousands of contacts and its mean is steady; at 300 kb separation on the same map those 48 cells are almost all zero, so the mean is a handful of counts divided by 48 — still noise — and widening the box does not help, because what is in it grows in proportion to nearly nothing. On a yeast map at 500 bp the coefficient of variation between 200 and 800 bins of separation is 6.8 raw, 1.1 at7 × 7, and 0.41 adaptive.This is serpentine’s criterion, not serpentine’s algorithm. Serpentine (Baudry et al., Bioinformatics 2020) is where the idea of holding the counts constant instead of the window comes from. Its own method grows regions and averages each, which makes the result a partition — the right shape for a differential analysis, and the wrong one for a viewer, because the partition is global and stochastic, so panning by one bin redraws every boundary and the map visibly reshuffles under the reader. Here the windows overlap and no partition is formed, so a cell reads the same wherever it sits in the viewport.
The width is set by the block’s as-read values while the mean is taken over whatever the derivation has made of them — serpentine’s own separation, and what makes the setting work under the O/E at all, since observed-over-expected is near 1 everywhere by construction and a window sized on it would be the same size everywhere. The target is a multiple of the block’s mean rather than an absolute count, because
scoremay have put balanced values in the cells.Picking it turns the O/E on, exactly as clicking the toggle would, and the
log2scale, the symmetric range and the diverging ramp all follow from the neutral of 1 that gives the result. That is the picture the setting is for: on raw counts all it can denoise is the distance decay that was never in question. Turning the O/E off again is left to the reader.The window is floored at
3 × 3and capped at a radius of 192. Blocks are capped at 16 M cells — the 4096² raster cap, so in practice it is available on anything the viewer draws. The cumulative tables are built for a band of rows at a time rather than for the block, so about 128 MB is resident whatever the block’s size; without that a 16 M block would allocate over half a gigabyte on the thread drawing frames. Time is roughly linear: 173 ms at 841², 328 ms at 1,200², 867 ms at 4 M and 3.9 s at 16 M. That last is a real freeze and the honest price of the cap being where it is; if it ever needs fixing the answer is the analysis thread rather than a lower cap. Above the cap the fixed3 × 3is used and the refusal is said twice — in the export caption, and in the view note under the heading, because a caption nobody reads until they export is no use to somebody looking at the screen. -
VCF opens as a lane of variants. Drop a
.vcf, or a.vcf.gzwith its.tbi, and the calls draw as boxes over the reference bases each record is about — with the symbolic ones at their real size, read out ofINFO/ENDandINFO/SVLEN, because<DEL>writes a singleNin REF whatever the deletion’s length and a forty-kilobase event drawn as a one-pixel tick is a wrong answer that looks like an answer.It was already being read, wrongly. The feature sniffer has no “none of mine” answer — its last line is
return 'bed'— and the row walker strips every#line, so a VCF reached it as a nameless eight-column table, was parsed as a BED, and was refused with “all 7 lines failed the contig, start, end test that every BED-like format begins with”: a message naming a format the reader never handed over.A substitution is drawn in the colour of the base it substitutes to. Over one locus with a FASTA and a BAM open, a heterozygous A>G now reads as one column in inks that agree — the sequence row writes a green
A, the pile puts orangeGmarks through half its reads, the depth row’s mark rises to half, and the variant row draws an orange box saying somebody called it. A record with no single base behind it — an insertion, a deletion, a structural variant, several ALTs at once — takes the grey anNis already drawn in, which already means no single-base answer. What separates an insertion from a deletion is extent, which is exact: a forty-kilobase deletion is a forty-kilobase box.A single base gets three pixels so it can be seen and clicked, and a lane stops resolving individuals the moment they are closer together than that — which is the rule that keeps a dense stretch from merging into one solid bar in whichever ink was drawn last. Collapsed, it says how many are there instead.
A VCF also says what genome it is on, exactly: its
##contiglines are achrom.sizeswritten inside the file, so a dropped VCF opens a browser over its real chromosomes rather than reporting each one as ending at its last variant. Reading part of a bgzipped one costs nothing new — it takes the same tabix road a bgzipped GTF does.The genotype columns are not drawn, and the lane says so: “1,284 variants · 2,504 samples not drawn”. A matrix of variants × samples is a second application’s worth of picture for a question a contact map is not asking, and a reader who brought a thousand-genome file should be told what they are looking at rather than left to infer it.
-
Alignment files open as two lanes. Drop a
.bamwith its.baior.csibeside it — or point at one by URL — and it lands as two rows over the one file: its depth, drawn like any other track (on any of the map’s four edges, in the 1D browser, with limits, an autoscale group, the ceiling drag, the readout and the exported figure, because it goes through the one funnel every lane’s values already come out of), and the reads themselves, in packed rows, once the view is narrow enough for a read to be more than a few pixels. A.bamcan also start a browser session on its own, since its@SQtable is every sequence and its exact length.Two rows rather than two bands of one lane, because they are two pictures with nothing in common — a curve against a value axis, and a stack of rectangles. Each is resized, reordered, hidden and closed on its own, so a reader who wants the depth without the pile closes the pile and keeps the link they had. Behind them is one open file and one parsed index, and the filters that decide which reads count — the mapping-quality floor and the duplicate switch — are written through to both, because a floor on the pile and not on the depth above it is two pictures of two different files under one name.
The pile lives only in the 1D browser, and that is geometry rather than policy: beside a heatmap the axis is an axis of matrix bins, where the finest unit is a bin and not a base, so there is nothing for a read to be drawn on. Too far out for a read to be wider than its own ink, or a lane dragged too short to hold a row, and it says zoom in rather than drawing a smear — the depth over the same reads is the lane beside it, and is drawn at every zoom.
A read is a rectangle only when nothing happened to it, and the exceptions are what anyone is looking at: a deletion is a rule through a gap because the read spans it and says it is absent, a spliced gap is a thinner rule because the read says nothing about what it skipped, an insertion is a tick between two bases with no width of its own, a soft clip is drawn faintly outside the aligned extent, and a mismatch is a coloured column — from
=/Xwhere the aligner wrote them, fromMDwhere it did not, and from a FASTA already open in this session where neither is there. The lane says which of the three it used, or that it had nothing to compare against, rather than drawing an agreement it did not check.A variant is coloured on both rows, where a FASTA is open. In the pile it is the mismatch mark it always was, one coloured column per read. On the depth row it is the share of each position that disagrees with the reference, drawn from the baseline up — so a heterozygous site is a mark half the height of the curve over it, and a homozygous one fills it. Above a fifth of the depth only, because under that every pileup is speckled with sequencing error; and only once a pixel is a handful of bases, because at a kilobase to the pixel the mark would be the genome’s background mismatch rate drawn everywhere.
The mark is drawn in a colour that is deliberately none of the four bases. It shipped as the sequence palette’s own T, which put one red in one column under two opposite claims — the reads disagree here on the depth, and this read has a T here in the pile — so a reader studying a T>C site would have read the same ink two ways an inch apart.
It costs no request: the residues come from a FASTA already open in the same worker, and a FASTA dropped after the BAM refreshes both lanes without a pan. It is not a variant call — substitutions and short deletions against whatever is loaded, with no base-quality filter, no strand-bias test and no score.
The controls are the pile’s, not a track’s. Colour by strand, by pair, by insert size, by where the mate went, or by mapping quality, with a key under the picker naming every category. It colours; it does not group — the rows are packed by position, so the categories are interleaved down the pile rather than banded, and a band per category is not built. Beside it the pair layout and the split-read switch, which are the pile’s alone; and on both rows the mapping-quality floor and the duplicate switch, which are about which reads count. All of it is stored with the session.
Four of the five reach the worker on the request that needs them; the layout is the one the worker never reads, so changing it redraws what is already in hand rather than fetching a window again.
A pair is drawn as a pair. Both ends of a template on one row, joined by a dotted thread across the part that was not sequenced — with
fragmentsandreadsbeside it in the card for a reader who wants the fragment as one bar, or a single-ended library packed record by record. The old switch drew a solid thread and packed by position alone, so at any real depth the two ends landed on different rows and the thread ran towards a mate that was not there. A change made while the previous window is still decoding replaces it — the test is against the running request’s controls, never against the drawn window’s, which between a change and the first landing is permanently the old one.Split reads are drawn and never counted. A supplementary alignment is a further piece of one molecule the aligner had to split across a breakpoint, so drawing it is how a reader sees the split and counting it would report one fragment as two. The split is what holds that: the depth is its own row on its own road, and the switch is not even in the key its samples are cached under, so turning it on costs the depth nothing at all. It is outlined rather than faded — faded already means “soft clip” here — and it is never joined to its primary as though it were a mate.
A click on a read opens a card and fetches nothing. The flags in words with the raw number beside them, the CIGAR, the clips, where the mate went, the insert size, the category it was coloured as. Every field is already in the pile the lane drew, which is why it answers at once and why it cannot disagree with the rectangle that was clicked — and the fields a BAM uses to mean “not recorded” are shown as that rather than as numbers.
This is the one format allowed to require a companion file, and that took an argument rather than a line of code. SPECS §13 listed required companion files among the explicit non-features, and §1 said C1 and C3 together mean the app can never require one. The scope of that rule is now written down: it protects the contact map’s opening path, which is what makes the no-preprocessing claim true, and it was never a claim about every byte the app reads. A BAM has no whole-file fallback that respects §12 at all — it is read by region and the index is the only thing that makes reading by region bounded — so the narrow rule replacing it is stated positively at §8.13: a format whose index is its only bounded read may refuse without one, and the refusal names the command. Two do:
samtools indexfor a file with no index beside it, andsamtools sortfor one that is not in coordinate order, because a name-sorted file handed to a region query answers with whatever was in those blocks, which is worse than refusing because it looks like an answer.Above what it can afford, the lane draws nothing and says why. There is no summary layer in a BAM, so depth over a window is a full read of the reads under it, and a window can be legitimately unanswerable. The forecast is made from the index alone, so a window too deep to draw is refused at zero data requests rather than discovered part-way through a fetch. Drawing the BAI linear index’s byte density there instead was considered and refused: it is compressed bytes per 16 kb rather than depth, wrong wherever read length varies, and absent entirely on CSI — it would be the first quantitative lane here that means something other than what it says.
Three decisions inside the depth itself, each a place a viewer can quietly disagree with
samtoolsand never say so. A CIGARNis not covered — it is reference the read is aligned across rather than over, and counting it draws an RNA-seq gene as a solid block. Secondary and supplementary alignments are excluded, since they are one molecule reported twice and they cluster exactly where the number is read most carefully. Duplicates are kept unless asked otherwise, because marking them is a pipeline’s decision and hiding them silently would disagree with every other reading of the same file.The reader is written here, with no new runtime dependency, for the reason
hic/format.tsandannotations/tabix.tsboth give at length.@gmod/bamis a devDependency and reads the fixtures alongside it, exactly ashic-strawreads the.hicones — and the first thing it caught was a record prefix off by four bytes in this repo’s reader and its fixture builder at once, which is precisely the mutual misunderstanding a round trip cannot see. -
Tabs can be called something. A tab is labelled by its file, which is the right default and a poor name for the work: four tabs reading
matrix_1000.mcooloff four runs are four identical labels whose only distinguishing part is the one they all share. Double-click a tab — or press F2, or find Rename this session in the palette — and type. Enter commits, Escape abandons, and an empty box hands the tab back to its file name, which is what makes a rename undoable.The name is part of the session rather than of the window, so it travels in the saved workspace, in the durable record and in a share link: a workspace of six tabs whose names did not come back is six tabs a reader has to open one at a time to tell apart. It is kept against the file, so it survives a change of resolution and a redirect into an entry of a collection, and opening a different file in the tab hands the label back — replacing the file is replacing the work. A tab with nothing open cannot be named, and the strip does not offer the gesture there rather than accepting a name it would have nowhere to keep.
The tooltip leads with the URI it has always led with, because a named tab is a tab whose file is no longer on screen anywhere else.
-
A saved workspace now carries four things it used to leave behind. Each was invisible in the same way — the file opened, the tabs came back, and one decision the reader had made was quietly the importing window’s instead. Every one of them is a decision that takes minutes to make and a moment to lose.
Which columns were folded, and how wide they were. A fold is a preference and lives beside the theme, which is exactly why it has to travel: a window that restores twelve tracks into an Annotations column the sender had dragged to 520 pixels, and puts it back at the default with Analysis unfolded over it, has restored the tracks and not the arrangement. All five columns, the Files panel included. A file that says nothing about a column leaves it alone.
How each companion of a deck was drawn — its transform, its ramp, its clip, the score it reads instead of the deck’s, and whether its chain was open. The deck’s own doctrine is that the view is shared and the presentation is not, and a deck of four replicates each clipped to its own depth is precisely what a deck is arranged for; restoring it onto the lead’s one clip is a legal deck, a plausible-looking one, and not the figure that was saved. Keyed by the file each map was showing, never by its slot — a slot id is minted per deck and names nothing tomorrow — which is the same key the parked lanes already use. One consequence worth stating: one file open twice in one deck has one look, and comes back on whichever of the two clips was set last.
What the session’s own map says about a track. Every other map’s lane was written down and the lead’s was not, so the two things a lane can say that a track cannot — I am not on this map and my range here is different — were sayable about every map in a deck except the one every session has. Both were losable in one gesture: taking a track off the lead is one press of its last lit edge, and every analysis result computed against a starred companion is placed there and taken off the lead. A deck of four then came back with the fourth’s tracks stacked on the first as well.
The overlay toggles, on the road that had never applied them. See below.
-
Companions’ scales are remembered across a reload, not only across a save. The same record the workspace carries is this session’s own memory, written on every change rather than on a save — so a reload, the durable restore after a browser restart, and an exported file all describe the same deck. Nudging four companions’ clips and closing the window used to lose all four, because a clip is not a navigation and nothing on the state road ever fired.
-
Expected (cooltools)beside the density on P(s). A Quantity choice on the contact-probability task. The default is unchanged — the probability density,HiContacts::distanceLaw()’snorm_p— and the other setting plots contacts at a separation over the bin pairs that could have held one, which is whatcooltools expected-cisreports and what a scaling exponent is compared against −1 or −1.5 as.They differ by exactly one denominator:
density / expected = pairs(s) / constant. The pair count is flat while the separation is short against the sequences and falls to zero at the longest one, so on the yeast demo the two are within 1 % below 10 kb, 13 % apart at 100 kb and a factor of fifty apart at 1.2 Mb — and on a mammalian map they agree over everything anybody fits.Verified end to end rather than by construction: pooled genome-wide over the yeast demo,
--quantity expectedreproducescooltools.expected_cisto 4e-6, which is the printing precision of the TSV. The quantity is cooltools’; the bins are the task’s ownfactor, so 1.2589 matcheslogbin_expected’s ten to the decade. What O/E and the pileups divide by is untouched — they reach the same numbers throughexpectedAt, never through the panel. -
Four text boxes on every curve, one per axis bound.
xfrom/to andyfrom/to, beside thelog x/log ytoggles. Empty means “fit the data”, so clearing all four is how you get back to the whole curve; a marquee fills them in, and typing in them is how two figures are set to the same window for a figure panel rather than dragged towards it. -
expectedin the score picker: the denominator of O/E, drawn. Every cell replaced by the mean of the block at that bin separation, with one scalar between each pair of sequences — a banded triangle fading from the diagonal and nothing else in it. It is in the picker rather than beside theO/Etoggle because it answers what the numbers are, not what they are compared to, and it overrides that toggle rather than composing with it: O/E of the baseline is 1 wherever it is anything, so the toggle greys and its tooltip says why.The reason to look at it is that an O/E is only as trustworthy as its denominator. At long separations the curve is a mean over few cells and it moves, so a ratio that swings out there is swinging in the model rather than in the biology — and until now there was no way to see which.
Drawn on a sequential ramp with no neutral, because a mean of counts is count-like; measured after whichever normalisation is in force, like the ratio it comes out of. Its id in a link is
baseline, notexpected, so a file that carries a/pixels/expectedcolumn of its own still reaches it — that column now readsexpected (column)in the list, which is the only way two entries could have carried one word. -
Pattern detection, and it is chromosight. A fifteenth task on the map card: loops, borders, hairpins, centromeres and stripes, found by correlating a template against a map detrended by its own distance law, with the smears suppressed and a Benjamini-Hochberg q-value on every call. It is a port of chromosight rather than a caller of this app’s own invention, and the difference is the whole reason it exists — §18.9 refused the second and gave a reason that the first answers: a background model and false-discovery control, both of them the published method’s.
It agrees with that package rather than resembling it. On
demo/yeast/Yeast_microC_G2M_1kb.coolat the loops preset,chromosight detectand this task both call 1,100 loops — the same 1,100, with scores agreeing to 5e-7.test/golden/analysis/chromosight.jsonis built by running the package itself and compares five stages separately: the undetectable bins, the detrended map, the correlation map, every call with its score, p-value and q-value, and the pileup. A single end-to-end check would have said “different loops”; these say which of five things moved.Four outputs, and they are different in kind on purpose. The loop set is a bedpe — a BED for the patterns that live on the diagonal — offered to the Annotations panel, so a call lands on the map it was called from before the reader has to decide whether to believe it. The pileup is every window it found, on chromosight’s own fixed 0–2 seismic scale, which is what lets it be laid beside a published figure. The spectrum is the distribution of the scores behind that pileup: a smooth shoulder running down to the threshold says the calling stopped where the threshold was rather than where the pattern did, and no list of coordinates ever says that. And a table of the strongest calls.
The pattern is a parameter rather than five tasks, and every threshold under the picker follows the picker — upstream tuned
pearson, the scan distance and the two window-quality percentages per template, so a form that baked one preset’s numbers into a box would report borders at twice the threshold chromosight uses. Set them myself takes them over, and says so in the provenance. -
APA has a fourth normalisation, and it is
chromosight quantify. The same anchors and the same averaging as the other three, with the control moved: the map is detrended by its own distance law before any window is cut, so a cell is observed-over-expected at that separation on that sequence — which is what chromosight’s pileups are, drawn on the same fixed scale. Three things follow and none of them is a detail: the window is the template’s size rather than the reader’s half-width, because the score under the picture is a correlation against those cells; the scan distance is the loop set’s own reach, which is whatcmd_quantifysets it to; and every anchor comes back, scored or not. It reports the per-anchor Pearson correlation, its p-value and its q-value beside the pileup, and the same score spectrum the detector draws — which is the half of the answer a pileup cannot carry, since two sets with the same mean window can be a hundred good loops and four hundred coordinates of which fifty are loops.Checked the same way: over the demo loop table, this and
chromosight quantifyproduce the same 1,506 scores and the same pileup, andtest/golden/analysis/chromosight_quantify.jsonis that package’s output. -
seismic, in the colormap picker. matplotlib’s own five stops, here becausepileup_plotdraws with it. A matrix output can now also fix its colour range rather than fitting one to the data — used by exactly the two pileups above, for the reason a fitted scale cannot serve: a pileup running from 0.98 to 1.02 and one running from 0 to 4 would otherwise be the same picture in the same colours, and the difference between them is the result. -
A contact map can be called something. A map is labelled by its file, which is the right default and a poor name for the work: two conditions dropped in to be compared are
mat_G1.mcool::1000andmat_G2M.mcool::1000, and every one of those characters is on the picture, in the readout under the pointer and in the exported figure — where the two words that would tell a reader what they are looking at areG1andG2M. Four replicates in a deck are four labels differing in a digit. Click a map’s name on the Contact maps card, or on a deck cell’s heading, and type. Enter commits, Escape abandons, and an empty box hands the map back to its file name, which is what makes a rename undoable — the Tracks card’s own control in both places, so a reader who has learned the gesture on a track has learned it here.The name is used everywhere the map is named: the card that lists it, the heading over its cell, the two bands of a turned comparison, the readout and the tooltip, the derived label on a ratio, and the caption on an exported figure. It is kept against the file, so it survives a change of resolution, and it travels in the workspace, in the durable record and in a share link beside the tab’s own name — a link carrying only the names of maps a recipient could open, since the address of a local one is a filename off the sender’s own disk. One consequence, which the deck’s saved looks already had: one file open twice in a session has one name.
-
The turned view of a comparison shows both files. A split map is the first file above the matrix diagonal and the second below it, and turning it kept only the upper triangle — so the second file, the entire reason the picture is split, disappeared the moment the reader turned it. It is now drawn as a second band mirrored beneath the first, the two meeting at one shared genomic axis: the same position is the same column in both, so they are read against each other at every place without the eye having to reflect one of them through a diagonal first. The distance up the side runs outwards from the axis in both directions, both bands are named on the picture, and a pointer in the lower band reads the lower band’s file. The pair is pinned by its middle, so a band cut deeper than its box — which is what a zoom in leaves — is clipped equally from both and the two files stay the same depth, where an edge would take the whole surplus off one of them. A ratio or a difference stays one band, its two triangles being the same numbers twice.
-
A map of several chromosomes now carries coordinates. A whole genome, or a selection of chromosomes, had the contig names and nothing else: an axis of several sequences has no single coordinate system, so it drew none of the ticks that assume one. It is now ruled one chromosome at a time, each measured from its own first base, with the boundaries the axis already draws saying where one ruler stops and the next begins — which is the question a reader is asking of a whole-genome map as much as of a single chromosome. The long chromosomes carry a position each and the short ones carry their name and nothing more, which is the rule the names themselves are already elided by, and a metagenome’s thousands of pixel-wide contigs carry none.
-
A deep scan of the source, and the thirty-one defects it confirmed. Ten review passes over
src/— the binary readers, the text parsers, rendering, the analysis tasks, session state, the network path, the UI, the deck, the workers, and documented-versus-actual behaviour — with every finding handed to a second reader whose brief was to refute it. Six claims died there. What survived is below, grouped by what it cost rather than by where it lived. Every one carries a test that fails without its fix.Wrong answers, silently. A version-9
.hicwas walked with version 8’s footer layout: v9 narrowed each per-chromosome scale factor from float64 to float32, and the walk still advanced twelve bytes per chromosome per vector instead of eight, overshooting the normalisation index. Depending on what the misplaced cursor met, the file either failed to open with a rawRangeErroror opened with its balancing weights silently missing. The reason it shipped is the more interesting half: the fixture writer emitted an empty expected-value section, so no test ever executed the loop, and thehic-straworacle reads v9 through a header pointer rather than by walking — so it agreed with us no matter what we did. The fixtures now write those sections populated, at each version’s widths, and the oracle fixture zeroes that pointer to force both readers down the same path.A tabix-indexed
.gtf.gzor.vcf.gzwas refused outright. Byte spans from the index start at BGZF block boundaries, which are not line boundaries, so the first and last lines of every fetched span were record fragments handed straight to the parsers — twenty-two lines, none of them parseable, and a message blaming the file. The seams are trimmed now, with the two edge cases that make the obvious rule wrong handled explicitly: a span starting at byte 0 begins on a real record, and a file whose last read reached EOF may legitimately end without a newline.A
drain()sized its buffer byContent-Lengthwhile reading a body the host had gzipped in transit — so the declared length was the compressed size and the file was cut to a fraction of itself and returned as complete. A body arriving shorter than promised was likewise accepted in silence. Of everything here this is the one that most deserved finding: silent truncation of scientific data that nothing downstream can detect.A scaled-region metaplot mapped each window’s base pairs uniformly across all of its columns, which is only correct when a feature happens to be exactly
body × steplong. Every other feature had its flanks compressed and its body stretched by a different amount, so an average over features that are genuinely aligned came out smeared. The mapping is piecewise now — flank, body, flank — and reference-point mode was verified bit-identical across four hundred randomised windows before and after.A BED domain vanished the moment you zoomed inside it: an
elementoverlay was culled by its bottom-right corner rather than tested for overlap, so any feature larger than the view drew nowhere. Abordermark was stroked along the diagonal, over the pixels it exists to point at. A gappedPeak’s signal statistics were read from columns 7-9, which in BED12+3 are thickStart, thickEnd and itemRgb. A headed annotation table with twelve or more columns had two unrelated columns decoded as BED12 blocks — now settled by checking that the blocks actually tile the feature rather than by counting columns, which also catches the headerless case a header could not have saved. And a per-bin profile carried across a resolution change was exported against the grid the session is on now rather than the one its values were computed on.Privacy. “Forget everything” cleared the stores and then let the very next state change write the durable record straight back. And a share link, which carefully strips private local filenames from two of the three places they appear, carried the third: a lane’s companion URI rode through untouched, so a link still spelled the sender’s filename into the fragment.
Things that stopped working, or never did. Closing the share dialog left
App.modalpointing at it forever, and every command guarded by “no dialog is open” — the palette, the shortcut sheet, About — was dead for the rest of the tab. Clearing a number box set it to the minimum, because the guard testedNumber.isFinite(Number(''))andNumber('')is0— the comment beside it said exactly that and the code did it anyway; the same dead guard was in four places, including a mapping-quality floor where the wrong answer was quieter still, since0legally means “keep everything”. A modal setaria-modal="true"and never trapped Tab, so a keyboard user walked out onto controls the backdrop was covering. Two notes shared one fixed corner, so a progress note covered the truncation warning that is specified to stay until dismissed — both are placed by a column now, which is also why the warning can outlive the job. Importing a workspace into a full window imported nothing and still overwrote the reader’s overlay preferences and column layout. In a deck at chromosome level the lock compared spans in bin sizes rather than in sequence lengths, so two panels showing the identical chromosome list were told they could not be locked and the deck blamed the files; a companion still opening was reported as having one resolution, and the false line never cleared; and the deck’s own map list ignored the names the reader had given. Every ordinary multi-chromosome.pairsimport was labelled unsorted, because the sortedness test assumed a global ordering the declaredchr1-chr2-pos1-pos2sort does not have — the repair pass it triggered was harmless but the label was not, and it is now raised only for disorder that ordering cannot explain. Ametacooler-runthat listed sixteen tasks could run twelve;overlapandmotifnow genuinely run, andmetaplotandtrackcorrare refused before a file is opened, because they need a bigWig reader the repo’s own layering confines to the annotations worker.Retention and lifecycle. A
HeatmapViewdestroyed while frozen kept re-schedulingrequestAnimationFrameon a detached canvas for the life of the tab; the ladder that took the freeze never gave it back on teardown, leaving a panel frozen for a read that would never report; and a deck navigating to a table went on parenting the previous, already-destroyed map — a canvas with a full-resolution backing store behind it. The bin file of a private HiC-Pro pair was never signed, so the half carrying every coordinate came back 403 while the half carrying the counts arrived. A Stop pressed while a task was in flight was read only before the step, so a run cancelled mid-step still finished, cached and adopted its result.Two smaller ones for completeness: an S3 bucket scope whose prefix contained an escaped character listed nothing, because the prefix was handed to S3 still percent-encoded; and a dropped-feature count was reported once per ticked track rather than once.
-
Every published demo failed to open. The picker offered three sessions and each one answered a click with not a workspace:
App.openWorkspaceFilereadskind: "metacooler.workspace"and refuses anything else, and the copies on the bucket are still written in the older single-session shape. The mock-community entry was a 404 on top of that — the bucket had moved that dataset underdemo/metagenomics/andui/examples.tswent on naming the old prefix.The catalogue is now four sessions, ordered as an argument rather than by dataset: one map with its tracks, then the same assembly with no map at all, then two maps side by side — a
.cooland a Juicer.hic, locked together, at the one window the two assemblies agree about to the base — and the metagenome last, because it is the file this viewer exists for and the one that needs the other three to be legible.demo/on disk is laid out the way the bucket is, so publishing is a small JSON sync rather than a second copy of 49 MB.Note that the fix is only half in this repository: the bucket has to be re-synced with
scripts/publish-demo.shbefore the site sees any of it. -
Sequence lengths inferred from a feature file were inflated by up to 85%. Open a browser on a GTF, a BED or a narrowPeak — a file that carries no lengths — and each sequence was rounded up to one significant figure, a step equal to the value’s own order of magnitude. Yeast IV (1.53 Mb) and VII (1.09 Mb) therefore both came out as exactly 2.00 Mb, 441 kb apart and indistinguishable; the genome total read 31% high; every chromosome ended in a long blank pannable margin; and the bar chart in the Sequences card was drawn to the wrong shape. The step jumped discontinuously at each power of ten, so the error was worst just above one.
The margin the rounding was written for is kept — a browser opened on a peak set should not appear to end at the last peak — at two orders of magnitude less: three significant figures, always strictly past the last feature, about a half-percent instead of a third. Yeast now totals 12.11 Mb against a true 12.07, where it read 15.99.
Two things the interface claimed and the code did not do are now true as well. The browser’s summary no longer says a sequence ends at its last feature, which it never did; and dropping a bigWig really does replace inferred lengths with real ones, as the Genome card has always offered — along with a bigBed, a BAM, a FASTA or a VCF that declares its contigs, any of which knows the answer exactly. It is adopted as well as loading the file as a track, never instead of it, and it can only ever replace a guess: an assembly that already knows its lengths is never overwritten. This works from the Tracks card’s own file button as well as from a drop, which is one gesture two inches from the other.
-
Hiding a curve recoloured the others. The legend filtered the hidden series out and the renderer then coloured by position in what was left, so hiding the first of three moved the second onto its colour and the third onto the second’s — every curve, its confidence band and its legend swatch, all shifting one step up the palette at once. Nothing said so, and it went into the PNG and the PDF. The most visible face of it was alt-click isolate, which hides all but one series and therefore always drew whichever you isolated in the first palette colour. Colours are now settled against the full list before anything is hidden. Affected P(s), the P(s) slope, the metaplot and the all-maps deck overlay — every figure that draws more than one curve.
-
The share dialog was clipped, cramped, and made you press a button. Its body asked for a width 4 rem greater than the panel containing it, and that panel is
overflow: hidden— so the right-hand edge, which is exactly where the copy and shorten buttons live, was cut off with no way to scroll to it. It also had no padding at all, so the text sat against the border. The link is now a box that scrolls by wheel and by drag rather than a field only a caret can move through, the copy control is the icon alone, and the link is put on the clipboard the moment it is generated, with a notification saying so — and again when the shortener replaces it, since that is what you will paste. Where the browser refuses the clipboard, which is every insecure origin, nothing claims success: the link is selected and a sentence says to copy it. -
The show/hide tooltip control was missing from the 1D browser. The matrix’s Tracks card builds its overlay row from the list of track options; the browser’s spelled six of them out by hand and
tooltipwas not among them. The flag was honoured — the browser reads it on every hover — but reachable only from thepshortcut or the command palette, never from the column the rest of the switches are in. -
An incomplete
.pairsfile is now said to be incomplete. A truncated BGZF — a cancelled upload, an interruptedbgzip, a partial copy — imported without complaint and drew a map of whatever had arrived. Every number in that map is true of the records that were there, which is exactly the problem: nothing on screen distinguishes this library has no contacts on chromosome XIII from this file stops before chromosome XIII.A BGZF file marks its own end with an empty member, so the two states are distinguishable and now are: a file that ends mid-member or without that marker is reported as incomplete, alongside any sequences that have nothing on their own diagonal — the visible form of the same fact. Both are carried into the written
.mcoolas well, since a cooler binned from an incomplete file is itself incomplete and will outlive the window that knew.Found on
demo/yeast/CH128.pairs.gz, which is cut off after chromosome XII and silently produced a map with four empty chromosomes. -
A bgzipped
.pairs.gzfailed to open in the browser withJunk found after end of compressed data.BGZF is a concatenation of gzip members, and the specification says a decoder walks them — Node’s does, and that is where it was verified. Chromium’s does not:DecompressionStream('gzip')stops after the first member and throws. Because BGZF’s end-of-file marker is itself an empty second member, this was true of every bgzipped file, not only a large one; the fixtures that passed were plain single-member gzip, which is the one shape that works either way.Members are now decompressed one at a time, which is what the format is. That turned out to cost nothing: taking the writer and reader off the
DecompressionStreamdirectly, rather than through aBloband aResponse, runs at 260 MB/s against theBlobform’s 34 MB/s — soannotations/tabix.ts, which reads.tbiand GTF spans the slow way, now shares the fast one. -
A Choose-files button in a framed page no longer does nothing at all. Anybody could already put this site in an
<iframe>, and in a cross-origin one the File System Access pickers are present and rejecting:typeof window.showOpenFilePickeranswers'function'and the call fails with aSecurityError. Every capability gate here tested thetypeof, andchooseFilesswallows a rejection because that is what a cancelled dialog looks like — so the button opened nothing and reported nothing, which is the one outcome worse than not offering it.handlesSupported,directoriesSupported,directoryPickerSupportedandchooseFilesnow ask whether this window’s top-level document is on another origin, and a frame that is takes the fallbacks Firefox and Safari have always taken: an<input type="file">for files, the snapshot shape for a directory. Dropping was never affected. A same-origin frame is deliberately not caught, since the pickers work there. -
Every analysis task failed on a HiC-Pro map with
error - name not defined!— HDF5’s own words, for a.matrixthat is not HDF5 at all. The two threads that mount a contact map had drifted apart: the reader worker grew the HiC-Pro road (two text files, read whole, no HDF5 anywhere) and the analysis worker, which mounts the same file a second time so a long task cannot block a pan, went on callingMetaCooler.opendirectly. The map was drawn and perfectly fine; the task said the file was broken.The fix is one road rather than a second branch.
worker/open.tsnow holds the knowledge of how many ways there are into a map, and the reader worker, the analysis worker and themetacooler-runCLI all go through it — so the CLI, which had the same defect and no one had hit it yet, is fixed by the same change rather than by remembering to. -
A task on a selection of cells measured every cell in the file. Because the analysis worker mounts the file itself, it never learned that an
.scoolhad a selection: it opened the pseudo-bulk, so two cells ticked and four thousand gave the same answer with nothing on screen to say so — silently wrong numbers, which is worse than the error above. The ticked set now travels with the target, and the worker’s mount is keyed on it as well as on the file, because re-ticking is a new map at the same URI and a cache on the name alone would answer the second question with the first one’s cells. -
A HiC-Pro matrix parsed nineteen times slower than it reads, and only ever in the application: 35 seconds for a 258 MB file where the same code measured 1.8. The scan shared its integer reader with the bin-file parser, and V8 will not optimise a function whose inline caches are still moving —
--trace-opt-verbosesaysdelaying optimization of scanSlice, IC changed, once per tick, forever — so having read a bin file first left the scan running interpreted from the first byte to the last. It reads its own digits now.The reason it was invisible is worth recording: the bins are parsed before the matrix in the application and never before it in a test, so every timing taken of this file was of the fast path. The fix carries the measurement and the reason beside it, because re-sharing the helper would restore the slow path silently.
Two things fell out of inlining the scan, both of which the old scanner had wrong and nothing had noticed: an exponent written with its sign (
1e+2, what%gand numpy print) stopped at the+and quietly kept the mantissa, so the pixel was 1 rather than 100; and a negative fractional value (-0.5) asked its integer part which way it went, got-0, and came out positive. -
A diagonal pixel was counting as both triangles, so the
MATRIX_FORMATa HiC-Pro file turned out to be written with was reported asmixedfor every real file — every real file has a diagonal. Caught against HiCExplorer’s own HiC-Pro test pair, which is upper-triangular and was labelledmixed; the comparison is now strict, and a file of nothing but its diagonal reports the default it would have been written as. -
A URL fragment is not part of a file name.
<matrix>#bins=<url>ends in.bed, so the name test sent the matrix through the genome door and drew its bin file as a feature track. Both places that take a name off a URL — the router and the worker — now drop the query string and the fragment, which the first was already half doing for presigned bucket URLs. -
Answering the
.scoolgate with small pieces asked the same question again, forever. Two causes, both worth naming. The retained remote source was keyed on the URL alone, so re-opening with the reader’s answer handed back the mount built for the other reading and discarded the answer; and the gate itself only knew about the source, which is still remote after that choice — choosing download it whole passed only incidentally, because that source stops being remote. The reading is now part of a mount’s identity, and the agreement is passed explicitly rather than inferred. -
2 << 30is not two gigabytes. The shift operators coerce to signed 32-bit, so the whole-object download limit evaluated to −2,147,483,648 and refused every file, including a 114 kB one, with “more than a browser tab can hold in memory”. Found by driving the built app in a browser rather than by a test, which is the kind of arithmetic no unit test was ever going to be pointed at. -
A bgzipped file with no index is read to its end, not to the end of its first block.
bgzipwrites a gzip member per block — roughly one per 64 KiB — and the whole-file road handed all of them to a singlegzipdecompressor, which is only correct on a platform that carries on past the first member’s end. The Compression Streams spec does not promise that; it says trailing data after a stream ends is aTypeError. So on a runtime that reads it that way, every.vcf.gz,.gtf.gzand.fa.gzwithout a.tbibeside it was refused outright with “Trailing junk found after the end of the compressed stream” — and a runtime that stopped quietly at the first member instead would have been worse, drawing the opening blockful and calling it the file.It now walks the blocks, which is the road the indexed path and every BAM already took, and asks nothing of the platform beyond
deflate-raw. A plain.gzis one member and not BGZF, so it still goes the way it did. -
A file and its index dropped together are now actually paired. The loader has always paired a
.tbiwith the file it indexes, andSession.dropped— which decides what a single file is — could only ever be asked about one file at a time, so every caller looped it over the batch and the loader was handed one file at a time. A plain drop of a bgzipped GTF and its.tbionto the window therefore read the GTF whole and drew the index as a refused row. The pairing rule now lives in one function that both roads go through. -
A re-opened local track keeps its index. A grant was remembered for the file and not for the sidecar, so a reload re-attached the
.gzand lost the.tbi, quietly going back to reading the whole file. Changing resolution did the same thing by a different route:carryrebuilt the source without it. The index now lives on the source rather than beside it, so every road that carries a source forward carries the index too. -
A derived track is no longer restored as a file that never existed. It is computed from two other rows and holds no bytes and no recipe the stored shape can carry, so it fell through to
{ kind: 'local' }and came back as a detached local row under the derived track’s label, offering to re-open a file that has never been on any disk. It is not persisted now, which is the smaller wrong until there is a stored shape that can name an operation and its operands. -
A BAM’s chromosomes are no longer described as guesses. Its
@SQtable carries a length per reference, so the lengths are exact — but the assembly was reported with originfeatures, which is whatlengthsAreExactreads, so the Genome card said they were inferred from the features and that each sequence ended at its last read. It has its own origin now, and the card says from the BAM’s @SQ header. -
The three places that list what this app opens now mention BAM. The landing page, the Tracks card and — the one worth spelling out — the landing page’s note that a file dropped with its index beside it is one track and not two.
-
The README no longer states a rule SPECS overturned. It said in two places that annotations were absent on a
.metacool— once in prose and once in the milestone table, where the Metadata card was still described as the only annotation such a file gets. §8.5 replaced that gate with an honest overlap report in MAGs some time ago. -
metacooler-run apa --window 8set the half-width and then died. The CLI reserves--windowfor the framing locus, andapadeclares a parameter of the same name — a half-width in bins.paramsForalready gives a task’s own parameters the flag where the two collide, but the framing read it anyway, so the run set the half-width correctly and then failed with “Unknown sequence label: 8”. One name, one meaning, decided by whether the task claims it. -
A second map to compare against was lost on a change of resolution, and silently: the first map is a complete picture on its own, so nothing on screen said the other one had gone, and the reader had to find and open it again at every zoom level. A comparison belongs to the file it is a comparison of, so opening another file puts it down — and a change of rung went through the same door, though it is not a change of subject. The second file now comes across, moved to its own rung at the new bin size: it has a ladder of its own, so
rep2.mcool::/resolutions/1000beside a 1 kb map is…/5000beside a 5 kb one, looked up in that file’s own entries. The reading carries with it — the operation, the depth match, and whether the triangle was hidden. A file with no rung at the new bin size cannot be compared there, two bin sizes in one square being a pixel that means one thing above the diagonal and another below it, and it is put down with the reason on the Contact maps card. -
Opening a saved workspace ignored the overlay toggles it carried. Every tab in the file has always recorded them — the dividers, the breaks, the labels, the ticks, the crosshair, the tooltip — and the one road that could not apply them was the road a workspace actually arrives on. A dropped single session and a pasted share link went through
Session.openConfig, which applies them; a workspace file and the offer after a browser restart go through the shell, which did not. So a figure saved with the dividers on and the crosshair off came back with whatever the importing window happened to be set to, silently, and looking exactly like the toggles had not been saved. The tab that ends up in front now decides, which is what a window-level preference carried per tab can honestly mean. -
P(s) is smooth again: the density is divided by the separation a bin covers, not by the width of the break. The curve had developed a sawtooth of a factor of two through the 10–30 kb decade, which buried the cohesin shoulder the derivative is read for.
It is an aliasing artefact and
HiContacts::distanceLaw()has it too. That break table isunique(round(1.1^n))in base pairs and was built for a pairs file, where a distance is continuous and the counts in a break really are spread over its whole width. A contact matrix is not like that: its separations sit on a lattice ofbinsize, and near 10 kb the breaks are about a kilobase wide — so on a 1 kb map one break holds two lattice points and the next holds one. Dividing both by their nominal widths makes the first almost twice as dense as the second, bin after bin. On the yeast demo the log-log slope of that curve moves by 0.83 between neighbouring points and swings between +1.7 and −2.9; the shoulder it hides is a change of about 0.26, so the artefact was an order of magnitude bigger than the feature.Dividing by
offsets × binsizeis the same quantity — a share of all cis contacts, per base pair — estimated on the lattice the data are actually on. It moves the slope by 0.087 between neighbours, and it agrees with the nominal width wherever a break is wide enough to hold several offsets, which is everywhere past about 50 kb at 1 kb bins. HiContacts says as much itself: its matrix branch prints “the P(s) curve will be an approximation” and its pairs branch does not.The golden now compares the numerator — the contacts in each break, break for break — against HiContacts exactly, which is where a real disagreement would show, and a new test asserts both roughness figures from the golden’s own numbers so the sawtooth cannot come back unnoticed.
-
The CLI can use a loop set again.
--pairs loops.bedpeparsed the file, filed it under its file name, and then set the task’s parameter to a path; every task that takes a feature or pair set looked under its own parameter key, found nothing, and reported “the set chosen is no longer loaded” about a file it had just read.TaskInputsis keyed by parameter key —analysis/job.tshas a whole paragraph about not indexing it any other way — and the CLI was the one venue that had no store to allocate ids in, so it invented a second convention and lost. Both work now: the name a reader typed still names the input, and the parameter key is an alias for it.
Changed
Section titled “Changed”-
The adaptive filter runs on the analysis thread above 1 M cells. It reads the block twice — once to measure every cell against its neighbourhood and once to average the capped field — and at the cell cap that is 7.4 s, which on the thread drawing frames is a frozen window rather than a hitch. Below the limit it is still done inline, which is the better trade there: 78 ms on a 353² view, and no flash of an unsmoothed map on the way to a smoothed one.
deriveBlockruns the pipeline to the point the smoothing would happen, hands the caller the arrays it would have used, and carries on with the unsmoothed field; the panel starts a job and derives again when the answer lands, at which point the step is a lookup. The correlation’s arrangement, for the correlation’s reason. While it waits the caption sayscounts, notsmoothed— a figure exported mid-wait must not claim a filter it does not have — the wait shows on the veil, a superseded job is cancelled, and a late answer to a superseded question is dropped rather than cached.Two callers never defer and compute inline at whatever it costs, because the alternative is not doing it at all: a panel with no analysis thread behind it, and either file of a compared pair, where two blocks would need two jobs and a half-finished pair would put a smoothed map over an unsmoothed one and call the result a ratio. The fixed radii never defer at any size — one incremental pass is not worth a round trip.
The job is not interruptible partway through, unlike the correlation’s, and that is deliberate: the filter’s passes are banded over rows and there is no consistent state to stop at inside one. The cancel is checked before the work as well as after, so a job cancelled while queued never starts and only the one already running is wasted.
-
The adaptive window no longer turns one hot cell into a rectangle. Reported from a yeast microC map at 500 bp: saturated hard-edged blocks, each with an isolated bright pixel at its centre and its edges lined up with discarded-bin stripes. Two causes, both ours. A mean has no defence against a single extreme value — and it had one until the centre went back into the kernel, which fixed the hollow squares and removed the only outlier rejection there was. And a box kernel’s impulse response is a box, so the cell raised every window that could reach it by the same amount and none of the next ones along. Measured: one cell at 400× its neighbourhood produced a 20 × 20 bin block peaking at 5.3 in
log2O/E, on a scale clipping at 0.84.A cap on leverage, before the averaging. Not a multiple of the local mean, which is the obvious rule and is wrong for this data: away from the diagonal the map is Poisson-sparse, so a window’s mean is 0.05 while its occupied cells hold 1, and every honest count is already twenty times the local mean. No fixed multiple both spares those and controls a spike — a tight one biases the sparsest part of the map downward, which is the part the filter is for. What the artefact is, is leverage: a cell shifts the mean of every window reaching it by
(value − mean) / n. So the cap is written on that — no cell may add more than half again to the mean of a window its own size — and it scales itself, tight in a 3 × 3 near the diagonal and two hundredfold in a sparse 21 × 21. The mean is taken without the cell itself, which is the one thing the zero-centre kernel had right: excluding the centre is correct for deciding whether a cell is an outlier and wrong for estimating what to draw there. The two passes do one each.And a shoulder on the kernel. The mean is now taken over three nested windows —
r/2,r,3r/2— and averaged, so the weights step down instead of falling off a cliff: the outer shell carries about 8% of the weight the centre does. The radii are chosen so this costs nothing in noise, an effective sample of 3.90·r² against the plain box’s 4·r² — the same estimate over half again as much ground.With both, a synthetic spike of 2000× no longer reaches the clip and the O/E’s mean stays at 1.00 in every separation band, so no bias was traded for the improvement. The cost is a second read of the block: 78 ms at 353², 300 ms at 841², 1.5 s at 4 M and 7.4 s at 16 M, about double what it was.
This reduces the artefact rather than removing it, and on real data it is still visible. The cap bounds a flagged cell’s damage by construction — the rule permits it to inflate its window by half, which is
log2 0.58on a scale clipping at 0.84 — and two stricter answers were tried and dropped: trimming the flagged cell out of the sum and the divisor, and replacing it with the mean of its eight neighbours. Both looked better on synthetic spikes and neither removed the artefact reproducibly on a real map, which says the detector rather than the response is what still needs work. The synthetic used to develop this evidently does not reproduce whatever the real one is. -
The adaptive window may now grow to radius 192, up from 64. At 64 it ran into the cap beyond about 200 bins of separation on a yeast map at 500 bp, so the cap rather than the target was choosing the width for exactly the cells with the least data. The search is binary, so this costs one more trial radius per cell and nothing else.
-
The despeckle now runs before the O/E division, and divides by a baseline averaged over the same windows. Both halves of this were needed to make the
smoothpicker do anything visible on an O/E map, which is the picture most readers of a sparse map are looking at.observedOverExpecteddraws a cell where nothing was observed as paper rather than as the strongest depletion on the ramp, and away from the diagonal most cells are that cell — 98% of them at 200 bins out on a yeast map at 500 bp. Computing the O/E first therefore handed the filter a field that was almost entirelyNaN, which no despeckle will touch, so every setting from3 × 3to7 × 7drew the same picture: a scatter of the 2% of cells that happened to catch a contact, at a mean of 45× expectation, over a blank field.Smoothed first, a zero cell with contacts around it is never blanked. That 98% falls to 83% at
3 × 3, 38% at7 × 7and 0% under every adaptive setting, and the mean of the band comes back to 1.00.The baseline is averaged over the same windows because dividing a smoothed map by an unsmoothed curve is biased: near the diagonal the decay is steep, so a window spanning it averages cells whose baselines differ several-fold and the ratio reads 24–32% enriched at radii 3 to 7 — an artefact of the filter and of nothing in the file. Matched, the bias sits in both halves and cancels. The curve itself is still estimated from the unsmoothed block, which is the half of the old ordering worth keeping: fitting it to pre-averaged cells would let each cell’s neighbours count twice toward the baseline it is compared against.
-
The despeckle keeps the cell’s own value. The kernel was
terra::focalwith a zero centre weight, on the argument that a cell disagreeing with all eight of its neighbours is evidence about the neighbourhood rather than about itself. That is right about rejecting outliers and wrong about a sparse map, where a lone contact is not an outlier but the only measurement there is: the zero-centre kernel replaced it with the mean of the eight zeros around it while spreading it into those eight, so every isolated count rendered as a hollow square with a hole where the contact was, and away from the diagonal most of the picture was made of those.It was also the wrong estimator for what the filter now feeds — a local rate that the O/E divides — since discarding the one cell being estimated at is both biased and needlessly noisy. Despeckling survives the change in all but the narrowest window: a wild pixel is diluted to a ninth of its excess at
3 × 3, a twenty-fifth at5 × 5, and to a few hundredths at the widths the adaptive settings reach. What is lost is the guarantee that a single wild cell is annihilated rather than suppressed; what is bought is that a single real cell is no longer annihilated either. -
Zooming a curve is a rectangle now, and it keeps working. Dragging across a P(s), a slope or any other drawer curve selects a window on both axes rather than only on the abscissa, and the marquee is drawn as the rectangle it selects. A drag that is deliberately flat still means what it looks like: an axis is only taken when the pointer travelled at least eight pixels along it, so a horizontal sweep is still a band and a mis-click is still a click.
And the marquee comes back. It lived in the plot body, and the rebuild after the first zoom replaced that body’s children — so the first drag worked and every drag after it selected an invisible rectangle, which read as the gesture having broken. Double-click still resets, now to the window the figure opened on rather than to the full extent.
The zoom is also no longer a clip of the series arrays. The window travels on the output as
xDomain/yDomainand the axes are drawn to it, which is what makes a window in y expressible at all — a point above the top of the frame is part of the line that leaves it, not a gap in the data — and the series are clipped to the frame in the PDF exactly as on screen. -
P(s) and its slope open on 4 kb – 1 Mb. Past the ligation artefacts at one end and inside the shortest chromosome at the other. The full extent was a bad picture of a good curve: the first break holds the diagonal and the first off-diagonal — two thirds of every cis contact on a 1 kb map — at a separation of 1 bp, so a log axis over all of it spent a third of its width on one point and squashed the decades anybody reads into the right-hand quarter. It is a default and not a limit: the marquee, the boxes and a double-click all move it. The ordinate follows, because an automatic y axis now fits what is inside the x window rather than the whole series.
-
APA has no Quality settings. Most anchors to use and Minimum coverage of a window are gone, not defaulted: every anchor is piled up and every window is summed. Both were quietly a change of subject rather than a budget — a cap makes the figure a pileup of whichever anchors sorted first, and a coverage floor drops exactly the windows in sparse regions, which are the ones a weak pileup is weak because of. Two runs could then differ because they had thrown away different windows, with nothing on the figure saying so. The accumulator counts per cell, so a half-empty window contributes what it has and no more.
-
APA’s half-width defaults to 12 bins, down from 25, and the note about the classic APA square is off the control.
-
P(s) is now the quantity HiContacts means by it. The contact-probability task plots the probability density — contacts at a separation over all cis contacts, then over the width of the separation bin in base pairs — which is
HiContacts::distanceLaw()’snorm_p. It used to plot the mean contact per pair of bins, which iscooltools expected-cis. Both are called P(s); they agree in shape at short range and part company as the separation approaches the length of a chromosome, where the pair count is heading for zero and the contact total is not, so an exponent read off the tail of one is not an exponent of the other. This changes the numbers every existing P(s) figure was drawn from, and the ordinate now saysContact probability density (per bp)to make that visible.The expected curve itself is unchanged and is still what observed/expected, the compartment chain and the pileups divide by — it just never was the curve on screen, and now it is not pretending to be.
It reproduces that package rather than resembling it. The separation is HiContacts’ gap distance — two bins k apart are
(k − 1)·binsizeapart there, so the diagonal and the first off-diagonal share the first break — and the bins are its break table, which turns out to be exactlyunique(round(1.1^n))and is regenerated from the task’s own factor rather than tabulated.test/golden/analysis/hicontacts_ps.jsonis built by running HiContacts’ own pipeline in R (scripts/hicontacts_ps.R, the first golden whose dependency is a program rather than a Python module), and every break position and density matches to 2e-6 on four chromosomes and on the four pooled.
Documentation of chromosomix v0.20.0 · built from 40ec1ec · source
