mirror of
https://github.com/kennethreitz/langchain.git
synced 2026-06-05 23:00:18 +00:00
update(llms/TGI): Allow None as temperature value (#10212)
Text Generation Inference's client permits the use of a None temperature as seen [here](https://github.com/huggingface/text-generation-inference/blob/033230ae667101d2d8d8bcd4952442fa348ef951/clients/python/text_generation/client.py#L71C9-L71C20). While I haved dived into TGI's server code and don't know about the implications of using None as a temperature setting, I think we should grant users the option to pass None as a temperature parameter to TGI.
This commit is contained in:
@@ -65,7 +65,7 @@ class HuggingFaceTextGenInference(LLM):
|
||||
typical_p: Optional[float] = 0.95
|
||||
"""Typical Decoding mass. See [Typical Decoding for Natural Language
|
||||
Generation](https://arxiv.org/abs/2202.00666) for more information."""
|
||||
temperature: float = 0.8
|
||||
temperature: Optional[float] = 0.8
|
||||
"""The value used to module the logits distribution."""
|
||||
repetition_penalty: Optional[float] = None
|
||||
"""The parameter for repetition penalty. 1.0 means no penalty.
|
||||
|
||||
Reference in New Issue
Block a user