aokitools commited on
Commit
5308842
·
verified ·
1 Parent(s): 9f3f1e0

Add or update README.md

Browse files
Files changed (1) hide show
  1. README.md +95 -0
README.md ADDED
@@ -0,0 +1,95 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language: ja
4
+ library_name: transformers
5
+ tags:
6
+ - continued-pretraining
7
+ - language-model
8
+ model-index:
9
+ - name: aokitools/japanese-laws-egov-instruct-202508051206
10
+ results: []
11
+ ---
12
+
13
+ # Experimental model in research stage
14
+
15
+
16
+ ## Quickstart
17
+
18
+ If you're using [Ollama](https://ollama.com/), run the following command first, then restart the Ollama app and select the newly added model.
19
+ ```shell
20
+ ollama pull hf.co/aokitools/japanese-laws-egov-instruct-202508051206
21
+ ```
22
+
23
+ If you want to remove it, run the following command:
24
+ ```shell
25
+ ollama list
26
+ ollama rm hf.co/aokitools/japanese-laws-egov-instruct-202508051206:latest
27
+ ollama list
28
+ ```
29
+
30
+
31
+ To use it from Python, use the following code.
32
+ ```python
33
+ import torch
34
+ from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
35
+
36
+ model_name = "aokitools/japanese-laws-egov-instruct-202508051206"
37
+
38
+ quant_config = BitsAndBytesConfig(
39
+ load_in_8bit=True,
40
+ llm_int8_threshold=6.0,
41
+ )
42
+
43
+ # load the tokenizer and the model
44
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
45
+ model = AutoModelForCausalLM.from_pretrained(
46
+ model_name,
47
+ torch_dtype=torch.float16,
48
+ device_map="auto",
49
+ quantization_config=quant_config,
50
+ )
51
+
52
+ # prepare the model input
53
+ prompt = "Give me a short introduction to large language model."
54
+ messages = [
55
+ {"role": "user", "content": prompt}
56
+ ]
57
+ text = tokenizer.apply_chat_template(
58
+ messages,
59
+ tokenize=False,
60
+ add_generation_prompt=True,
61
+ enable_thinking=True # Switches between thinking and non-thinking modes. Default is True.
62
+ )
63
+ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
64
+
65
+ # conduct text completion
66
+ generated_ids = model.generate(
67
+ **model_inputs,
68
+ max_new_tokens=256
69
+ )
70
+ output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
71
+
72
+ # parsing thinking content
73
+ try:
74
+ # rindex finding 151668 (</think>)
75
+ index = len(output_ids) - output_ids[::-1].index(151668)
76
+ except ValueError:
77
+ index = 0
78
+
79
+ thinking_content = tokenizer.decode(output_ids[:index], skip_special_tokens=True).strip("\n")
80
+ content = tokenizer.decode(output_ids[index:], skip_special_tokens=True).strip("\n")
81
+
82
+ print("thinking content:", thinking_content)
83
+ print("content:", content)
84
+ ```
85
+
86
+
87
+ This model is a continual pretraining of [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B).
88
+
89
+ ## Training details
90
+ - Base model: Qwen3-1.7B
91
+ - Tokenizer: QwenTokenizer
92
+
93
+ ## License
94
+ - Apache 2.0 + Alibaba Qianwen License
95
+