Glonce

AWS's Strands Robots syncs robot data through Hugging Face Storage Buckets

infrastructureThe Glonce editorsHugging Face🔥57Glonce rating57Impact55Novelty38Relevance85Surprise28Players60Credibility85
AWS's Strands Robots syncs robot data through Hugging Face Storage Buckets

AWS's Strands Robots is an open source (Apache 2.0) SDK that wraps robot abstractions, simulation, and the LeRobot stack as AgentTools that plug into a Strands agent. An earlier post in this Hugging Face blog series introduced the SDK: the Robot() factory, recording a demonstration in simulation, running a policy, and deploying the same agent code to a physical SO-101. This second post assumes that setup and follows the data in the other direction, from the first recorded frame back to a deployed policy, using Hugging Face Storage Buckets, a mutable, non-versioned, Xet-backed object storage repository type that Hugging Face announced in March 2026. A bucket sits in the same hf:// namespace as a dataset repository and uses the same hf CLI a Strands Robots setup already has installed.

The problem it addresses: running the record, train, and deploy loop once is easy. Running it every day is not, because each training run has copied the whole dataset to the GPUs before starting, and syncing a versioned dataset repository turns every append into a retained, versioned commit, so a later change re-uploads everything. LeRobot's dataset format already backs more than 90,000 datasets and models from over 8,000 publishers on the Hugging Face Hub, according to LeRobot Project Pulse, and a Strands Robots recording uses that same format, so anything already built to read LeRobot data can read it without conversion.

The guide walks through a single agent handling all four stages. Robot('so100') records a LeRobotDataset through a shared DatasetRecorder; sync_dataset_to_bucket(...) syncs the finished recording into a Storage Bucket; stream_dataset(...) reads that same dataset back over the Hub for training, decoding camera video on the fly with no local copy; and the trained checkpoint deploys back to the same Robot object by changing one keyword argument, from mode='sim' to mode='real'. In the post's own minimal example, sim = Robot('so100'), agent = Agent(tools=[sim]), and a single natural-language prompt tells the agent to record a demo and sync it to a named bucket. Training then loops over sim.stream_dataset(...).dataloader(batch_size=64) without downloading the dataset to disk first.

Step 1 records into a bucket rather than a versioned dataset repository, because collection needs the opposite of version history: somewhere to overwrite bytes in place, using the permissions a Hugging Face workspace already has, with no identity and access management (IAM) roles, cross-origin resource sharing (CORS) rules, or separate upload service to configure. The walkthrough's example prompt builds a simulated world, adds a red cube and a camera, and records 60 steps with a placeholder mock policy, a stand-in that produces joint actions without a trained model. The post says this yields a structurally valid dataset, not a useful one, and syncs the result with sync_dataset_to_bucket('/tmp/cube_pick', 'my-org/robot-fave'), which returns a bucket URI shaped like hf://buckets/{bucket}/{run_id}, with run_id defaulting to the dataset's directory name. The same sync is available on an open recorder through DatasetRecorder.sync_to_bucket(), or automatically when stop_recording(bucket=...) is called; a separate push_to_hub() call still produces the versioned, published artifact, in the same underlying format. On physical hardware, LeRobot's own lerobot-record command-line tool drives an SO-101 leader-follower pair through the same recording, and the resulting on-disk dataset syncs to a bucket with the same call, for instance naming the run explicitly with run_id='run-021'.

Step 2 covers what a repeated sync actually costs. Storage Buckets are backed by Xet, which deduplicates uploads at the byte level through content-defined chunking: chunk boundaries follow the file's content, so inserting a few bytes changes only the chunk it falls into rather than shifting every boundary after it. A plain overwrite has no such protection; it resends the whole object regardless of how much changed. By Hugging Face's own measurements, content-defined chunking cuts the data transferred per upload by about four times across the Hub, and on Enterprise plans, billing is based on the deduplicated footprint rather than the raw upload size. Hugging Face's own bucket benchmark shows this on a single 500 MB file: re-uploading after changing 1% of its bytes moved 5.5 MB, changing 5% moved 27.5 MB, and changing 10% moved 55 MB. How much a given dataset benefits depends on its file layout, and the Strands Robots recorder uses LeRobot's own layout: episodes land in Parquet data shards and per-camera MP4 video shards that roll over to a new file once the current one fills, at LeRobot's defaults of 100 MB for a data shard and 200 MB for a video shard. A sync after a day of recording therefore uploads only the new trailing shards plus the one shard that was partway full at the previous sync, not the whole dataset, and Xet repeats the deduplication on every later sync of the same bucket.

Running the minimal, simulation-only version of the loop needs Python 3.12 or newer on Linux or macOS (Apple Silicon is supported through the MuJoCo backend), a Strands-compatible model provider for the agent's reasoning (Amazon Bedrock, the Anthropic API, OpenAI, or a local Ollama), and the package strands-robots[sim-mujoco,lerobot]>=0.5.1, which pulls in LeRobot 0.6.1 or newer plus datasets, av, and torchcodec. That combination runs the whole loop on a laptop, but only with the mock policy. A useful trained policy needs the advanced path: a Hugging Face account with a write token, huggingface-hub>=1.6.0,<2.0.0 plus hf auth login, an SO-101 leader-follower pair (or another LeRobot-supported robot) with calibration files under ~/.cache/huggingface/lerobot/calibration/, an NVIDIA GPU for local vision-language-action inference or a GPU cluster to train at scale, and lerobot[training] installed, without which trainer.train() returns an error result rather than a checkpoint. A runnable companion notebook for the whole walkthrough is at examples/notebooks/05_streaming_data_loop.ipynb. The captured text breaks off partway through the Step 2 deduplication example, so the post's own worked detail for streaming into training and deploying the checkpoint is not available beyond the overview above.

Key facts

  • Hugging Face Storage Buckets, a mutable, non-versioned, Xet-backed object storage repository type announced in March 2026, sit in the same hf:// namespace as dataset repositories and use the same hf CLI, serving as the overwrite-in-place layer between recording robot demonstrations and training on them.
  • Xet's content-defined chunking deduplicates uploads at the byte level; per Hugging Face's own measurements it cuts data transferred per upload by about four times across the Hub, and on a 500 MB benchmark file, changing 1% of the bytes and re-uploading moved 5.5 MB, changing 5% moved 27.5 MB, and changing 10% moved 55 MB.
  • The Strands Robots recorder writes LeRobot's own file layout, with Parquet data shards capped at 100 MB and per-camera MP4 video shards capped at 200 MB, so a sync after a day of recording uploads only the new and partially filled shards rather than the whole dataset.
  • LeRobot's dataset format already backs more than 90,000 datasets and models from over 8,000 publishers on the Hugging Face Hub, so a Strands Robots recording needs no conversion for anything already built to read LeRobot data.
  • A single Strands agent runs the whole loop: Robot('so100') records a LeRobotDataset, sync_dataset_to_bucket syncs it to a bucket, stream_dataset streams it back over the Hub with no full download to train, and the trained checkpoint redeploys to the same Robot object by switching one keyword argument from mode='sim' to mode='real'.

Why it matters

Running an agent-driven record, train, and deploy loop for robot learning once is straightforward. Running it every day is where the cost shows up: each training run has copied the whole growing dataset to the GPUs before starting, and syncing a versioned repository re-uploads everything, since every append becomes a retained commit. This post pairs that daily loop with Hugging Face Storage Buckets, a mutable, non-versioned, Xet-backed object storage type Hugging Face announced in March 2026, and Xet's byte-level, content-defined-chunking deduplication, so a sync after a day of new robot recordings uploads only the shards that actually changed. That turns a cost that grows with the whole dataset into one that grows with the day's new footage, which matters to anyone iterating on physical-AI data collection rather than running the loop once.

Who it affects

The guide targets developers already using AWS's Strands Robots, the open source (Apache 2.0) SDK that exposes robot abstractions, simulation, and the LeRobot stack as AgentTools inside a Strands agent, who have worked through the first post in this series covering the Robot() factory and a single record-and-deploy pass on a physical SO-101. It is aimed at the point where someone wants to move from a one-off recording session to a continuous collection-and-training cycle: deciding which episodes to keep, when a scene has drifted enough to re-record, and which checkpoint replaces the one running on the arm. Because a Strands Robots recording uses LeRobot's own dataset format, it also reaches the wider LeRobot ecosystem: more than 90,000 datasets and models from over 8,000 publishers already use that format on the Hugging Face Hub, per LeRobot Project Pulse.

How to use it

The minimal path runs entirely on a laptop: Python 3.12 or newer on Linux or macOS (Apple Silicon works through the MuJoCo backend), a Strands-compatible model provider such as Amazon Bedrock, the Anthropic API, OpenAI, or a local Ollama, and the package strands-robots[sim-mujoco,lerobot]>=0.5.1, which pulls in LeRobot 0.6.1 or newer along with datasets, av, and torchcodec. That path uses a mock policy, so it exercises the full loop without producing a policy worth deploying. Training a real one needs the advanced path: a Hugging Face account with a write token and huggingface-hub>=1.6.0,<2.0.0 plus hf auth login, an SO-101 leader-follower pair (or another LeRobot-supported robot) with calibration files under ~/.cache/huggingface/lerobot/calibration/, an NVIDIA GPU for local vision-language-action inference or a GPU cluster to train at scale, and lerobot[training] installed, since skipping it makes trainer.train() return an error result rather than a checkpoint. Strands Robots itself is Apache 2.0 licensed. A runnable companion notebook sits at examples/notebooks/05_streaming_data_loop.ipynb.

How solid is it

This is a vendor how-to, posted to Hugging Face's blog under the amazon organization's channel as a direct follow-up to an earlier post in the same series, not an independent test or review. No individual author, engineer, or spokesperson is named anywhere in the piece. The deduplication numbers, an about four-times cut in data transferred per upload across the Hub, and the 500 MB benchmark showing 5.5 MB, 27.5 MB, and 55 MB moved at 1%, 5%, and 10% byte changes, are described as Hugging Face's own measurements on generic files, not a benchmark run specifically on robot demonstration data. The post connects them to the robot case through LeRobot's Parquet and video shard sizes rather than a separate robot-specific measurement. Storage Buckets themselves were already announced in March 2026, so the underlying feature predates this post; what is new here is the worked integration with Strands Robots and LeRobot, backed by runnable example code and a companion notebook rather than prose claims alone.

Risks and caveats

The captured text ends partway through the Step 2 deduplication example, so the post's own detail on Step 3 (streaming the dataset back to train) and Step 4 (deploying the checkpoint to hardware) is not available beyond the high-level description given earlier: stream_dataset() reads the dataset back over the Hub without a full local copy, and the trained checkpoint redeploys by switching mode='sim' to mode='real'. The default, laptop-only path records with a mock policy that produces a structurally valid dataset but not a useful one, so the loop as shown out of the box does not by itself yield a working grasping policy. No customer, company, or production deployment is cited as using this workflow; it is a self-contained walkthrough. No GPU model or cloud instance type is named, only 'an NVIDIA GPU' or 'a GPU cluster' generically, and no price is given for Storage Buckets usage itself, only the relative transfer savings from deduplication.