Skip to content

Conversation

@CharlieFRuan
Copy link
Member

@CharlieFRuan CharlieFRuan commented Aug 23, 2024

Change

  • [Model] Add Phi3.5-mini #555
    • Adds the newly release Phi3.5-mini, adding the following model_ids to our prebuilt model list:
      • Phi-3.5-mini-instruct-q4f16_1-MLC (4k KVCache)
      • Phi-3.5-mini-instruct-q4f32_1-MLC (4k KVCache)
      • Phi-3.5-mini-instruct-q4f16_1-MLC-1k (1k KVCache)
      • Phi-3.5-mini-instruct-q4f16_1-MLC-1k (1k KVCache)
    • Old Phi-3-mini models are still kept

TVMjs

@CharlieFRuan CharlieFRuan merged commit 4ce15d8 into mlc-ai:main Aug 23, 2024
jingyi-zhao-01 pushed a commit to jingyi-zhao-01/web-llm that referenced this pull request Dec 8, 2024
### Change
- mlc-ai#555

### TVMjs
- Updated to current head:
apache/tvm@1518008
  - Main change is apache/tvm#17251
- This is needed for WASMs compiled after
apache/tvm#17257 is merged (e.g. Phi-3.5). TVM
global functions that returns bool need this PR to run correctly (e.g.
`AcceptToken()` in BNFGrammar) in runtime.
- However, these are backward compatible to WASMs compiled prior to this
PR. Tested with Phi-3 (old WASM) running grammar.
atebites-hub pushed a commit to atebites-hub/web-llm that referenced this pull request Oct 4, 2025
### Change
- mlc-ai#555

### TVMjs
- Updated to current head:
apache/tvm@1518008
  - Main change is apache/tvm#17251
- This is needed for WASMs compiled after
apache/tvm#17257 is merged (e.g. Phi-3.5). TVM
global functions that returns bool need this PR to run correctly (e.g.
`AcceptToken()` in BNFGrammar) in runtime.
- However, these are backward compatible to WASMs compiled prior to this
PR. Tested with Phi-3 (old WASM) running grammar.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant