kreasof-ai/Homunculus-Project
Long term project about a custom AI architecture. Consist of cutting-edge technique in machine learning such as Flash-Attention, Group-Query-Attention, ZeRO-Infinity, BitNet, etc.
This project offers an experimental, custom-designed AI architecture, akin to a 'slow thinking' mechanism, that can internally iterate and 'rethink' its cognitive process. It takes various forms of input, including vision and text embeddings, and produces refined outputs like generated text or image detokenizations, guided by an internal confidence score. This is for AI researchers and machine learning engineers looking to build highly efficient and sophisticated large language models with unique internal reasoning capabilities.
No commits in the last 6 months.
Use this if you are a machine learning researcher or engineer developing cutting-edge AI models and need an architecture that can self-correct and deepen its understanding through internal iterations.
Not ideal if you are a practitioner looking for an out-of-the-box, pre-trained model for immediate application rather than a foundational architecture for deep customization.
Stars
8
Forks
1
Language
Python
License
AGPL-3.0
Category
Last pushed
Oct 15, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/kreasof-ai/Homunculus-Project"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Lightning-AI/litgpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
liangyuwang/Tiny-DeepSpeed
Tiny-DeepSpeed, a minimalistic re-implementation of the DeepSpeed library
catherinesyeh/attention-viz
Visualizing query-key interactions in language + vision transformers (VIS 2023)
microsoft/Text2Grad
🚀 Text2Grad: Converting natural language feedback into gradient signals for precise model...
huangjia2019/llm-gpt
From classic NLP to modern LLMs: building language models step by step. 异æ¥å›¾ä¹¦ï¼šã€Š GPT图解 å¤§æ¨¡åž‹æ˜¯æ€Žæ ·æž„å»ºçš„ã€‹-...