Christopher Akiki

@cakiki.bsky.social

1300 Followers 74 Following

data and research at @lichess.org

t-SNE plot of animal engravings.
image quilt of all o'reilly media books, sorted by hue.
small multiple photo of 1360 oreilly animals
UMAP plot
scatter plot of 1 million personas from https://huggingface.co/datasets/nvidia/Nemotron-Personas-USA
Clifford attractor
3 by 3 grid of networks color-mapped with "plasma" and with edge-bundling. Lung looking structure.
3 by 3 grid of the same plot in different colormaps. Plot is an edge-bundled connectivity graph. Color maps are, left-to-right and top-to-bottom: viridis, plasma, inferno, magma, cividis, binary, copper, coolwarm, managua.
Organic looking scatterplot. Blob in the bottom left. Hedgehog looking.
Three scatterplots of colorful points. titles = ['Color Space', 'Text Space', 'Image Space'] subtitles = ['Embeddings of color features', 'Text embedding of color names', 'Image embeddings of color swatches']
Scatterplot of a document corpus with cluster information in the form of colors and cluster labels. Includes labels like computing, mental health, religion, etc.
organic looking network representing the internet BGP nodes. Inferno colormap.
Correct version of the plot in the quoted post. Graph with an organic like structure of the BGP network.
organic looking graph of the BGP nodes of the internet. black and white
very colorful scatterplot of player deaths in mario maker levels. "inferno" colormap. some level elements highlighted.
screenshot from the embeddings atlas repo
box art from 1996 version of CNC RED ALERT
We observe that the merge list of LLAMA, LLAMA 3, GEMMA, and MISTRAL contain clusters of redundant merge rules. For instance, in the LLAMA 3 merge list, we see the sequence of merges _ the, _t he, and _th e, as well as _ and, _a nd, and _an d. Because the merge path for every token is unique, it is impossible for more than one of these merges to ever be used, and we empirically verify this by applying the tokenizer to a large amount of text. We find that this is an artifact of the conversion from sentencepiece to Huggingface tokenizers format. To construct the merge list, the conversion algorithm naively combines every pair of tokens in the vocabulary, and then sorts them by token ID, which represents order of creation. While this is functionally correct, because the redundant merges are not products of the BPE algorithm (i.e., they do not actually represent the most-likely next-merge), we need to remove them to apply our algorithm. To do this, we do some simple pre-processing: for every cluster of redundant merges, we record the path of merges that achieves each merge; the earliest path is the one that would be taken, so we keep that merge and remove the rest. As an aside, this means that a tokenizer’s merge list can be completely reconstructed from its vocabulary list ordered by token creation. Given only the resulting token at each time step, we can derive the corresponding merge.
world map with points of interest
Scatterplot with 4.5M data points. Inferno colormap. Looks somewhat organic with neat clusters.
Scatterplot of the Aya dataset. 250K points in a floral / broccoli like pattern. Inferno color map.
Black and white image of a strange attractor.
scatter plot of a UMAP projection. magma colormap. quite clustered. looks almost celestial.
Scatterplot. Starbursty and elongated clusters. Magma colormap.
connectivity graph of the previous scatter plot. grayscale. edges bundled.
2d scatterplot in the viridis palette. Different blobs and clusters. Looks very starbursty in some places and very organic in others.
Scatterplot of the BigScience ROOTS corpus
Scatterplot of many points using a fire-theme palette on a black background. The points cluster in small groups. The general shape is circular, as is common of TSNE embeddings.