transformers-bnlang

Bnlang transformers — pipeline API on top of onnxruntime-bnlang (and optionally onnxruntime-genai-bnlang for fast LLM generation).

v0.1.0 supports text generation.

Unofficial third-party binding. Not affiliated with Hugging Face or Microsoft.

এই README-এর বাংলা সংস্করণ — README.md

Quick start

import "transformers-bnlang" as bt;

var gen = bt.pipeline("text-generation", "./models/Qwen2.5-0.5B-Instruct");
var out = gen.run("Hello", {
    max_new_tokens: 100,
    temperature:    0.7,
    top_k:          50,
    do_sample:      true
});
print(out["generated_text"]);
gen.close();

Two engines, one public API

// Default: our hand-rolled native loop. Runs ANY decoder-only ONNX
// model (Qwen, Llama, Mistral, Phi, …). Reads standard HF layout
// (config.json, tokenizer.json, onnx/model.onnx).
bt.pipeline("text-generation", "./models/Qwen2.5-0.5B-Instruct");

// Fast path: routes through onnxruntime-genai (Microsoft's
// specialized LLM runtime). ~10× faster, requires a model dir
// with genai_config.json. Same run/chat/close API.
bt.pipeline("text-generation", "./models/qwen-0.5b-dml-int4",
            { engine: "onnxruntime-genai" });

Exports

Pipeline options

Run-time options (gen.run(text, opts))

Chat

var out = gen.chat([
    { role: "system", content: "You are a helpful assistant." },
    { role: "user",   content: "What is the capital of Bangladesh?" }
], { max_new_tokens: 64 });
print(out["new_text"]);

Chat templates are picked from the architecture descriptor (chatml for Qwen, llama2 for Llama / Mistral). Custom templates land in 0.2.

Supported architectures (config.json model_type → descriptor)

Add a new model family with one entry — see lib/architectures.bnl and the register_architecture(name, descriptor) escape hatch.

Build (local)

# Windows
.\build.ps1        # cmake configure + build  ->  build/windows-x64/transformers-bnlang.dll

# macOS / Linux
./build.sh

Layout

bnl.json                 manifest (main + targets map)
CMakeLists.txt           build config
CMakePresets.json        one preset per platform

lib/
  index.bnl              public API (English + Bangla re-exports)
  pipeline.bnl           pipeline("text-generation", ...) dispatch + engine switch
  tokenizer.bnl          AutoTokenizer.from_pretrained / from_file
  architectures.bnl      descriptors for Qwen2, Llama, Mistral (extensible)
  chat_template.bnl      chatml + llama2 templates
  generation.bnl         KV-cache loop + sampling (the "our loop" engine)

src/                     C++ — BPE tokenizer + argmax/sample (excluded from published tarball)
  bnl/plugin.h           C ABI contract
  main.cpp               bnl_load + argmax_last + sample_last natives
  bpe.{h,cpp}            byte-level BPE tokenizer (~700 LOC)
  external/json.hpp      vendored nlohmann/json single-header

build/<triple>/          cmake output (gitignored)
test/
  smoke.bnl              dtype + tokenizer round-trip

License

MIT. Underlying ORT and GenAI prebuilts are MIT-licensed by Microsoft. See NOTICES.md for third-party attribution — in particular nlohmann/json, which is statically embedded.

MIT License

Copyright (c) 2026 Bnlang | Mamun

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to furnish persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Third-party notices — transformers-bnlang

This package contains code from third parties. Two distinct relationships:

1. Statically embedded — code compiled into our binary .dll/.so/.dylib. The original license requires we ship the copyright + permission notice alongside the binary. That's the full text below for nlohmann/json.

2. Transitive runtime dependency — code loaded at runtime via the onnxruntime-bnlang plugin (which itself dynamically links to ONNX Runtime). Attribution + upstream pointer is below.

nlohmann/json — statically embedded in the binary

• Project: JSON for Modern C++
• Upstream: https://github.com/nlohmann/json
• Version vendored: 3.11.3 (at src/external/json.hpp in our source tree)
• License: MIT
• Copyright: Copyright (c) 2013–2023 Niels Lohmann
• SPDX: MIT

Used by the BPE tokenizer to parse tokenizer.json. Because the library is header-only, its compiled code ends up inside our shipped binary.

Full license text

MIT License

Copyright (c) 2013-2023 Niels Lohmann

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to furnish persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

ONNX Runtime — transitive (via onnxruntime-bnlang)

• Project: Microsoft ONNX Runtime
• Upstream: https://github.com/microsoft/onnxruntime
• License: MIT
• Copyright: Copyright (c) Microsoft Corporation

This package depends on onnxruntime-bnlang, which downloads and dynamically links to ORT's prebuilt binaries. We do not bundle ORT itself; see onnxruntime-bnlang/NOTICES.md for the full attribution.

The full MIT license text is available at https://github.com/microsoft/onnxruntime/blob/main/LICENSE