krohling/nl-act
Integrating Natural Language Instructions into the Action Chunking Transformer for Multi-Task Robotic Manipulation
This project helps roboticists program industrial or research robots to perform complex manipulation tasks by simply providing natural language instructions. You input descriptions like "grasp the red cube" or "stack the blocks," and the system outputs the precise control commands needed for the robot to execute those actions. It's designed for robotics engineers or researchers who need to quickly teach robots new multi-step tasks without extensive coding for each scenario.
No commits in the last 6 months.
Use this if you need to enable a robot to understand and execute varied manipulation tasks based on diverse natural language commands, without retraining the robot for every new phrasing or slight task variation.
Not ideal if you are looking for a plug-and-play solution for consumer robotics or if you do not have access to a GPU-enabled computing environment for training.
Stars
9
Forks
1
Language
Python
License
MIT
Category
Last pushed
Dec 23, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/krohling/nl-act"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
kyegomez/RT-X
Pytorch implementation of the models RT-1-X and RT-2-X from the paper: "Open X-Embodiment:...
kyegomez/PALI3
Implementation of PALI3 from the paper PALI-3 VISION LANGUAGE MODELS: SMALLER, FASTER, STRONGER"
chuanyangjin/MMToM-QA
[🏆Outstanding Paper Award at ACL 2024] MMToM-QA: Multimodal Theory of Mind Question Answering
lyuchenyang/Macaw-LLM
Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration
Muennighoff/vilio
🥶Vilio: State-of-the-art VL models in PyTorch & PaddlePaddle