claudeindex
Plugin

tokenization

Text tokenization for LLMs including HuggingFace Tokenizers and SentencePiece. Use when training custom tokenizers or handling multilingual text.

Installation

1

Add the marketplace

/plugin marketplace add tianhao909/AI-Research-SKILLs-cn
2

Install plugins

/plugin

Run these commands in Claude Code to add this plugin to your environment. The marketplace must be added before you can install its plugins.

Repository & Links

Plugin Source

View Plugin Code

GitHub Repository

tianhao909/AI-Research-SKILLs-cn

Details & Metadata

2

Skills