kyegomez/HeptapodLM
An Implementation of an Transformer model that generates tokens non-linearly all at once like the heptapods from Arrival
This project offers a novel way for researchers and deep learning practitioners to generate text not in a traditional line, but across a two-dimensional grid, much like a crossword puzzle or a complex diagram. It takes a matrix of tokens (like words or characters) as input and outputs a similarly structured matrix, where meaning is conveyed through spatial relationships. This is for those exploring experimental text generation beyond conventional sequential models, seeking to represent and create content with inherent multi-directional meaning.
No commits in the last 6 months.
Use this if you are a deep learning researcher or practitioner interested in exploring unconventional, non-linear text generation architectures inspired by the spatial arrangement of meaning.
Not ideal if you need a readily trainable model for established natural language processing tasks that require sequential text generation.
Stars
10
Forks
—
Language
Python
License
MIT
Category
Last pushed
Nov 11, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/kyegomez/HeptapodLM"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in...
kyegomez/LongNet
Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
pbloem/former
Simple transformer implementation from scratch in pytorch. (archival, latest version on codeberg)
NVIDIA/FasterTransformer
Transformer related optimization, including BERT, GPT
kyegomez/SimplifiedTransformers
SimplifiedTransformer simplifies transformer block without affecting training. Skip connections,...