[Bug]: model_name and token_usage not returned in callback when invoked via agent

[nitin302]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

traceloop/openllmetry#2166 has code example and more details which confirms this is a bug in langchain code.

Error Message and Stack Trace (if applicable)

No response

Description

I am using langchain to invoke agents via agent_executor.invoke with Azure Open AI. However, model_name and token_usage is not returned via callback. This is required to send telemetry via openllmetry.

System Info

aiohappyeyeballs==2.4.3
aiohttp==3.10.9
aiosignal==1.3.1
alembic==1.13.3
annotated-types==0.7.0
anthropic==0.36.0
anyio==4.6.0
appdirs==1.4.4
appnope==0.1.4
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
asgiref==3.8.1
asttokens==2.4.1
async-lru==2.0.4
attrs==24.2.0
Babel==2.15.0
backoff==2.2.1
bcrypt==4.2.0
beautifulsoup4==4.12.3
bleach==6.1.0
blessed==1.20.0
boto3==1.34.116
botocore==1.34.116
build==1.2.2.post1
cachetools==5.5.0
certifi==2024.8.30
cffi==1.16.0
charset-normalizer==3.4.0
chroma-hnswlib==0.7.3
chromadb==0.4.24
ci-info==0.3.0
clarifai==10.7.0
clarifai-grpc==10.7.1
click==8.1.7
cloudpickle==3.0.0
cohere==5.10.0
colorama==0.4.6
coloredlogs==15.0.1
comm==0.2.2
configobj==5.0.9
configparser==7.1.0
confluent-kafka==2.5.0
contextlib2==21.6.0
crewai==0.36.0
dataclasses-json==0.6.7
datasets==3.0.1
debugpy==1.8.1
decorator==5.1.1
defusedxml==0.7.1
Deprecated==1.2.14
dill==0.3.8
diskcache==5.6.3
distro==1.9.0
docstring_parser==0.16
durationpy==0.9
einops==0.8.0
embedchain==0.1.114
et-xmlfile==1.1.0
etelemetry==0.3.1
executing==2.0.1
fastapi==0.110.0
fastavro==1.9.7
fastjsonschema==2.19.1
filelock==3.16.1
fitz==0.0.1.dev2
flatbuffers==24.3.25
fqdn==1.5.1
frozenlist==1.4.1
fsspec==2024.9.0
gguf==0.10.0
google-api-core==2.21.0
google-api-python-client==2.114.0
google-auth==2.35.0
google-auth-httplib2==0.2.0
google-auth-oauthlib==1.2.0
google-cloud-aiplatform==1.70.0
google-cloud-bigquery==3.26.0
google-cloud-core==2.4.1
google-cloud-resource-manager==1.12.5
google-cloud-storage==2.18.2
google-crc32c==1.6.0
google-resumable-media==2.7.2
google_search_results==2.1.0
googleapis-common-protos==1.65.0
gptcache==0.1.44
gpustat==1.1.1
grpc-google-iam-v1==0.13.1
grpcio==1.66.2
grpcio-status==1.62.3
gunicorn==23.0.0
h11==0.14.0
httpcore==1.0.6
httplib2==0.22.0
httptools==0.6.1
httpx==0.27.2
httpx-sse==0.4.0
huggingface-hub==0.25.2
humanfriendly==10.0
idna==3.10
importlib_metadata==8.4.0
importlib_resources==6.4.5
inflection==0.5.1
inquirerpy==0.3.4
instructor==1.3.3
interegular==0.3.3
ipykernel==6.29.4
ipython==8.24.0
isodate==0.6.1
isoduration==20.11.0
jedi==0.19.1
Jinja2==3.1.4
jiter==0.4.2
jmespath==1.0.1
joblib==1.4.2
json5==0.9.25
jsonpatch==1.33
jsonpointer==3.0.0
jsonref==1.1.0
jsonschema==4.22.0
jsonschema-specifications==2023.12.1
jupyter-events==0.10.0
jupyter-lsp==2.2.5
jupyter_client==8.6.1
jupyter_core==5.7.2
jupyter_server==2.14.0
jupyter_server_terminals==0.5.3
jupyterlab==4.2.0
jupyterlab_pygments==0.3.0
jupyterlab_server==2.27.1
kubernetes==31.0.0
langchain==0.2.1
langchain-chroma==0.1.4
langchain-cohere==0.1.8
langchain-community==0.2.0
langchain-core==0.2.0
langchain-openai==0.1.7
langchain-text-splitters==0.2.1
langgraph==0.0.50
langsmith==0.1.133
lark==1.2.2
llvmlite==0.43.0
lm-format-enforcer==0.10.6
looseversion==1.3.0
lxml==5.3.0
Mako==1.3.5
markdown-it-py==3.0.0
MarkupSafe==3.0.1
marshmallow==3.22.0
matplotlib-inline==0.1.7
mdurl==0.1.2
mistral_common==1.4.3
mistune==3.0.2
mmh3==5.0.1
monotonic==1.6
mpmath==1.3.0
msgpack==1.1.0
msgspec==0.18.6
multidict==6.1.0
multiprocess==0.70.16
mypy-extensions==1.0.0
nbclient==0.10.0
nbconvert==7.16.4
nbformat==5.10.4
nest-asyncio==1.6.0
networkx==3.3
nibabel==5.3.0
nipype==1.8.6
notebook==7.2.0
notebook_shim==0.2.4
numba==0.60.0
numpy==1.26.4
nvidia-ml-py==12.560.30
oauthlib==3.2.2
onnxruntime==1.19.2
openai==1.51.2
openlit==1.24.0
openpyxl==3.1.3
opentelemetry-api==1.27.0
opentelemetry-distro==0.48b0
opentelemetry-exporter-otlp==1.27.0
opentelemetry-exporter-otlp-proto-common==1.27.0
opentelemetry-exporter-otlp-proto-grpc==1.27.0
opentelemetry-exporter-otlp-proto-http==1.27.0
opentelemetry-instrumentation==0.48b0
opentelemetry-instrumentation-aiohttp-client==0.48b0
opentelemetry-instrumentation-aiohttp-server==0.48b0
opentelemetry-instrumentation-alephalpha==0.33.2
opentelemetry-instrumentation-anthropic==0.33.2
opentelemetry-instrumentation-asgi==0.48b0
opentelemetry-instrumentation-asyncio==0.48b0
opentelemetry-instrumentation-bedrock==0.33.2
opentelemetry-instrumentation-boto3sqs==0.48b0
opentelemetry-instrumentation-botocore==0.48b0
opentelemetry-instrumentation-chromadb==0.33.2
opentelemetry-instrumentation-cohere==0.33.2
opentelemetry-instrumentation-dbapi==0.48b0
opentelemetry-instrumentation-fastapi==0.48b0
opentelemetry-instrumentation-google-generativeai==0.33.2
opentelemetry-instrumentation-groq==0.33.2
opentelemetry-instrumentation-grpc==0.48b0
opentelemetry-instrumentation-haystack==0.33.2
opentelemetry-instrumentation-httpx==0.48b0
opentelemetry-instrumentation-jinja2==0.48b0
opentelemetry-instrumentation-lancedb==0.33.2
opentelemetry-instrumentation-langchain==0.33.2
opentelemetry-instrumentation-llamaindex==0.33.2
opentelemetry-instrumentation-logging==0.48b0
opentelemetry-instrumentation-marqo==0.33.2
opentelemetry-instrumentation-milvus==0.33.2
opentelemetry-instrumentation-mistralai==0.33.2
opentelemetry-instrumentation-ollama==0.33.2
opentelemetry-instrumentation-openai==0.33.2
opentelemetry-instrumentation-pinecone==0.33.2
opentelemetry-instrumentation-qdrant==0.33.2
opentelemetry-instrumentation-redis==0.48b0
opentelemetry-instrumentation-replicate==0.33.2
opentelemetry-instrumentation-requests==0.48b0
opentelemetry-instrumentation-sagemaker==0.33.2
opentelemetry-instrumentation-sqlalchemy==0.48b0
opentelemetry-instrumentation-sqlite3==0.48b0
opentelemetry-instrumentation-system-metrics==0.48b0
opentelemetry-instrumentation-threading==0.48b0
opentelemetry-instrumentation-together==0.33.2
opentelemetry-instrumentation-tornado==0.48b0
opentelemetry-instrumentation-tortoiseorm==0.48b0
opentelemetry-instrumentation-transformers==0.33.2
opentelemetry-instrumentation-urllib==0.48b0
opentelemetry-instrumentation-urllib3==0.48b0
opentelemetry-instrumentation-vertexai==0.33.2
opentelemetry-instrumentation-watsonx==0.33.2
opentelemetry-instrumentation-weaviate==0.33.2
opentelemetry-instrumentation-wsgi==0.48b0
opentelemetry-propagator-aws-xray==1.0.2
opentelemetry-proto==1.27.0
opentelemetry-sdk==1.27.0
opentelemetry-semantic-conventions==0.48b0
opentelemetry-semantic-conventions-ai==0.4.1
opentelemetry-util-http==0.48b0
orjson==3.10.7
outlines==0.0.46
overrides==7.7.0
packaging==23.2
pandas==2.2.3
pandocfilters==1.5.1
parameterized==0.9.0
parso==0.8.4
partial-json-parser==0.2.1.1.post4
pathlib==1.0.1
pexpect==4.9.0
pfzy==0.3.4
pillow==10.4.0
platformdirs==4.2.2
posthog==3.7.0
prometheus-fastapi-instrumentator==7.0.0
prometheus_client==0.20.0
prompt_toolkit==3.0.48
propcache==0.2.0
proto-plus==1.24.0
protobuf==4.25.5
prov==2.0.0
psutil==6.0.0
psycopg2-binary==2.9.9
ptyprocess==0.7.0
pulsar-client==3.5.0
pure-eval==0.2.2
py-cpuinfo==9.0.0
pyairports==2.1.1
pyarrow==17.0.0
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycountry==24.6.1
pycparser==2.22
pydantic==2.9.2
pydantic_core==2.23.4
pydot==3.0.2
Pygments==2.18.0
PyJWT==2.9.0
PyMuPDF==1.24.6
PyMuPDFb==1.24.6
pyparsing==3.1.4
pypdf==4.3.1
PyPika==0.48.9
pyproject_hooks==1.2.0
pysbd==0.3.4
python-dateutil==2.9.0.post0
python-dotenv==1.0.0
python-json-logger==2.0.7
python-multipart==0.0.9
python-rapidjson==1.20
pytz==2024.2
pyxnat==1.6.2
PyYAML==6.0.2
pyzmq==26.0.3
ray==2.37.0
rdflib==7.0.0
redis==4.6.0
referencing==0.35.1
regex==2023.12.25
requests==2.32.3
requests-oauthlib==2.0.0
requests-toolbelt==1.0.0
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rich==13.9.2
rpds-py==0.18.1
rsa==4.9
s3transfer==0.10.3
safetensors==0.4.5
schedule==1.2.2
schema==0.7.5
scikit-learn==1.5.0
scipy==1.14.1
Send2Trash==1.8.3
sentencepiece==0.2.0
setuptools==75.1.0
shapely==2.0.6
shellingham==1.5.4
simplejson==3.19.3
six==1.16.0
sniffio==1.3.1
soupsieve==2.6
SQLAlchemy==2.0.35
stack-data==0.6.3
starlette==0.36.3
sympy==1.13.3
tabulate==0.9.0
tenacity==8.5.0
terminado==0.18.1
threadpoolctl==3.5.0
tiktoken==0.7.0
tinycss2==1.3.0
tokenizers==0.20.1
torch==2.4.0
torchvision==0.19.0
tornado==6.4
tqdm==4.66.5
traceloop-sdk==0.33.2
traitlets==5.14.3
traits==6.3.2
transformers==4.45.1
tritonclient==2.50.0
typer==0.12.5
types-python-dateutil==2.9.0.20240316
types-requests==2.32.0.20240914
typing-inspect==0.9.0
typing_extensions==4.12.2
tzdata==2024.2
uri-template==1.3.0
uritemplate==4.1.1
urllib3==2.2.3
uuid6==2024.7.10
uvicorn==0.30.1
uvicorn-worker==0.2.0
uvloop==0.20.0
vllm==0.6.2
watchfiles==0.24.0
wcwidth==0.2.13
webcolors==1.13
webencodings==0.5.1
websocket-client==1.8.0
websockets==13.1
wrapt==1.16.0
xxhash==3.5.0
yarl==1.14.0
zipp==3.20.2

[dosubot]

Hi, @nitin302. I'm Dosu, and I'm helping the LangChain team manage their backlog. I'm marking this issue as stale.

Issue Summary:

• A bug is reported where model_name and token_usage are not returned in the callback when using agent_executor.invoke with Azure Open AI.
• This information is essential for sending telemetry data through openllmetry.
• The issue persists even after updating to the latest version of LangChain.
• No further comments or developments have been made on this issue.

Next Steps:

• Please confirm if this issue is still relevant with the latest version of LangChain. If so, you can keep the discussion open by commenting here.
• If there is no response, this issue will be automatically closed in 7 days.

Thank you for your understanding and contribution!