Search: github.com/tokenium | Heykuki News

HK

Heykuki News

Top New Best Ask Show Jobs

Top New Best Ask Show Jobs

1.

Tokenium: my new protocol to solve decentralized exchange flaws

github.com/tokenium

9 years ago

1 points

2.

Ask HN: Anyone have experience with open-source alternatives to Parse?

14 years ago

1 points

3.

Show HN: HuggingFace – Fast tokenization library for deep-learning NLP pipelines

github.com/huggingface

6 years ago

168 points

4.

Show HN: Convert a number to an approximated text expression

github.com/tokenmill

6 years ago

114 points

5.

TokenMonster: Ungreedy tokenizer, outperforming tiktoken by 35%

github.com/alasdairforsythe

3 years ago

10 points

6.

Show HN: Accelerated Text – open-source Natural Language Generation platform

github.com/tokenmill

6 years ago

6 points

7.

superfly/tokenizer: HTTP proxy that injects 3rd party credentials into requests

github.com/superfly

5 months ago

4 points

8.

Show HN: Email Sleuth – Find and Verify Emails (Rust CLI)

github.com/tokenizer-decode

a year ago

iLikeFunctional

3 points

9.

EmailSleuth – Find and Verify Emails, Rust Based (SMTP, Chrome Drivers)

github.com/tokenizer-decode

a year ago

iLikeFunctional

3 points

10.

Tokenization: A comprehensive deep dive into the world of tokens

github.com/SumanthRH

3 years ago

3 points

11.

A curated list of resources dedicated to Natural Language Generation

github.com/tokenmill

6 years ago

3 points

12.

A go library for tokenizing text

github.com/dannav

10 years ago

2 points

13.

Show HN: Personal AI gateway for OpenClaw – tokenomics

github.com/rickcrawford

4 months ago

2 points

14.

Show HN: New CLI tool to count tokens locally

github.com/zahidcakici

5 months ago

2 points

15.

Tokenizations alignment library for Rust and Python, written in Rust

github.com/tamuhey

6 years ago

2 points

16.

Show HN: Tokenusage – Rust CLI that tracks Claude Code/Codex tokens 214x faster

github.com/hanbu97

4 months ago

1 points

17.

TokenTamer A proxy that reduces LLM token usage through context compression

github.com/borhen68

13 days ago

1 points

18.

Free open source AI cost tracker – pip install tokenbudget

github.com/AIMasterLabs

3 months ago

1 points

19.

Claude hack – Don't waste you token where it's not needed

github.com/justinjamesmathew

a month ago

1 points

20.

Tokenmonster: Determine tokens to optimally represents a dataset

github.com/alasdairforsythe

3 years ago

1 points

21.

Atomic SWAR Token Buckets

github.com/jart

4 years ago

1 points

22.

Supercharge your search with tokenization

github.com/ashubham

8 years ago

1 points

23.

Show HN: Actual Claude Tokenizer

tokenizer.robkopel.me

2 months ago

3 points

24.

Show HN: My own tokenization method in Python, open to feedback

github.com/YasTouchedMySpaggett

3 years ago

2 points

25.

Show HN: Stateless TikToken and Unix-Dictionary GitHub URL Shortener

selfdotsend.com

7 months ago

1 points

26.

Show HN: ai-tokenizer – 5-7x faster than tiktoken with AI SDK support

coder.github.io

8 months ago

1 points

27.

Show HN: LLaMA tokenizer that runs in browser

github.com/belladoreai

3 years ago

115 points

28.

Approximation of the Claude 3 tokenizer by inspecting generation stream

github.com/javirandor

2 years ago

9 points

29.

GPT-4-Tokenizer-Visualizer

github.com/functorism

3 years ago

6 points

30.

Show HN: FlashTokenizer – 10x faster C++ tokenizer for Python

github.com/NLPOptimize

a year ago

5 points