claudeindex
Plugin

tokenization

Text tokenization for LLMs including HuggingFace Tokenizers and SentencePiece. Use when training custom tokenizers or handling multilingual text.

Installation

1

Add the marketplace

/plugin marketplace add zechenzhangAGI/AI-research-SKILLs
2

Install plugins

/plugin

Run these commands in Claude Code to add this plugin to your environment. The marketplace must be added before you can install its plugins.

Repository & Links

Plugin Source

View Plugin Code

GitHub Repository

zechenzhangAGI/AI-research-SKILLs

Details & Metadata

2

Skills