v0.19.0

Major Changes

• November Binary Update by @martindevans in #962
• Custom Sampler Stages by @martindevans in #961
• Add Microsoft.Extensions.AI support for IChatClient / IEmbeddingGenerator by @stephentoub in #964

Bug Fixes

• Non-Null Default SamplingPipeline by @martindevans in #973

New Contributors

• @stephentoub made their first contribution in #964

Full Changelog: v0.18.0...v0.19.0

v0.18.0

Major Changes

• Split platform-specific binaries for NuGet backends by @m0nsky in #957

Other Changes

• Updates to ContributingGuide for latest llama.cpp repo by @scritch1sm in #953
• Fix README chat session example by @easis in #956

New Contributors

• @easis made their first contribution in #956

Full Changelog: v0.17.0...v0.18.0

v0.17.0

Important: The CUDA packages for 0.17.0 exceeded the maximum size for a nuget package. This means some of the 0.17.0 packages are not available until a new way is to deploy those packages. If you need one of the missing packages, use 0.16.0 instead.

Major Changes

• October binary update by @martindevans in #940

Bug Fixes

• Fix: Increase Vulkan detection timeout to 12 seconds by @LSXPrime in #931
• Specify file open permissions needed as read only. by @scritch1sm in #943
• Fix llama_get_logits_ith Null Handling by @martindevans in #920
• Fix CUDA compile error by @martindevans in #924

Other Changes

• Add LLama2 Chat Session example with a custom templator by @asmirnov82 in #938

New Contributors

• @LSXPrime made their first contribution in #931
• @scritch1sm made their first contribution in #943

Full Changelog: v0.16.0...v0.17.0

v0.16.0

Major Changes

• August 2024 Binary Update by @martindevans in #905
• LLamaEmbedder 2.0 by @martindevans in #902

Bug Fixes

• Fixed constrained output (Grammar) support in BatchedExecutorGuidance by @webitube in #904
• Fixed Issue where Model with Grammar constrains only generates one output in Example by @L-Wehmschulte in #908

Other Changes

• Remove OpenCL & add Vulkan to readme by @m0nsky in #907
• Bump Swashbuckle.AspNetCore from 6.6.2 to 6.7.3 by @dependabot in #912
• Bump BenchmarkDotNet from 0.13.12 to 0.14.0 by @dependabot in #913
• Bump Microsoft.NET.Test.Sdk from 17.10.0 to 17.11.0 by @dependabot in #914
• Bump Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation from 8.0.7 to 8.0.8 by @dependabot in #916
• Bump Microsoft.KernelMemory.Abstractions and Microsoft.SemanticKernel.Abstractions by @dependabot in #915

New Contributors

• @webitube made their first contribution in #904
• @L-Wehmschulte made their first contribution in #908

Full Changelog: v0.15.0...v0.16.0

v0.15.0 - Llama 3.1

Major Changes

• Llama 3.1 update binaries by @martindevans in #874
• Microsoft.KernelMemory version 0.68+ compatibility fix by @SpaceAntelope in #862

Bug Fixes

• Fixed missing [DllImport] for llama_encode by @martindevans in #864
• Safer Vulkan System Info by @martindevans in #885
• Fixed LLamaTimings by @martindevans in #881

Other Changes

• Fixed Incorrect Exception Docs Comments by @martindevans in #883
• Deprecate InferenceParams sampler configuration properties by @martindevans in #871
• Updated Semantic Kernel and Kernel Memory by @martindevans in #880
• Update README.md by @sangyuxiaowu in #872

New Contributors

• @SpaceAntelope made their first contribution in #862

Full Changelog: v0.14.0...v0.15.0

0.14.0 - Gemma2

Major Changes

• July 2024 binaries by @martindevans, @SignalRT & @m0nsky in #833, #843
• Vulkan support by @m0nsky in #794, #797, #799
• Generic Prompt Formatter by @patrick-hovsepian in #787
• Async implementation of LLamaExecutors by @asmirnov82 in #834
• Include debug symbols with NuGet packages by @dpmm99 in #851
• Allow continuation in Instruct and Interact executors by @dpmm99 in #852

Bug Fixes

• Fixes Typo in IContextParamsExtensions by @SerialKicked in #806
• Fix possible file locks due to race conditions during build & test by @m0nsky in #836
• Fixed CI Out Of Disk Space by @martindevans in #846
• Fix CUDA detection on WSL by @m0nsky in #847

Other Changes

• Remove Beam Search by @martindevans in #783
• Batched Beam Search by @martindevans in #796
• Remove OpenCL Backend. by @martindevans in #782
• BoolQ Benchmark by @martindevans in #802

New Contributors

• @patrick-hovsepian made their first contribution in #787
• @SerialKicked made their first contribution in #806
• @dpmm99 made their first contribution in #851

Full Changelog: v0.13.0...v0.14.0

0.13.0

Major Changes

• June 2024 Binary Update by @martindevans in #751
• Add Rosetta2 Binaries by @abhiaagarwal in #755
• Optimize the native library loading. by @AsakusaRinne in #688
• Add experimental auto-download support. by @AsakusaRinne in #692
• Upload native libraries to huggingface when release new version. by @AsakusaRinne in #749
• Batch Queueing by @martindevans in #761
• BatchedExecutor LLava by @martindevans in #770
• StreamingTextDecoder Special Token Support by @martindevans in #777

Documentation

• Updated READMEs. by @Lamothe in #742
• Add documentation for the change of native library loading. by @AsakusaRinne in #747
• Add OAI demo link by @sangyuxiaowu in #778

Bug Fixes

• KernelMemory bug fix by @zsogitbe in #726
• Fixes for website when running on case-sensitive filesystems. by @Lamothe in #753

Other Changes

• Exposed basic timing information from llama.cpp by @martindevans in #736
• Less Sampler Allocations by @martindevans in #735
• Several updates to web project by @Lamothe in #718
• Conversation Generate All Logits by @martindevans in #743
• Removed .NET 7 now that it's reached EOL. by @Lamothe in #741
• BatchedExecutor Double Buffering by @martindevans in #748
• Minor BatchedExecutor Example Improvements by @martindevans in #750
• LLava API Improvements by @martindevans in #757
• SemanticKernel HomeAutomation Example by @zsogitbe in #769

New Contributors

• @Lamothe made their first contribution in #718
• @sangyuxiaowu made their first contribution in #778
• @abhiaagarwal made their first contribution in #755

Full Changelog: v0.12.0...v0.13.0

0.12.0 - LLama3

Major Changes

• May 2024 Binary Update (Take 2) by @martindevans in #712
• Logging interceptor by @martindevans in #649
• fix: add cuda llava native libraries. by @AsakusaRinne in #647
• Added Whisper.net x LLamaSharp examples for Speech Chat by @Lyrcaxis in #656
• Extension LLava with in memory images by @zsogitbe in #653
• Llava Initial approach to clear images by @SignalRT in #664
• BatchedExecutor Save/Load by @martindevans in #681
• Interruptible Async Model Loading With Progress Monitoring by @martindevans in #702
• LLava Async Loading by @martindevans in #703
• Optional IHistoryTransform added to ChatSession.InitializeSessionFromHistoryAsync by @Norne9 in #711
• Implement context shifting in executor base by @ksanman in #714
• Llama Text Templater by @martindevans in #715
• Make LLamaKvCacheView Safe by @martindevans in #721

Documentation

• Removed ContextSize from most examples by @martindevans in #663
• Add issue templates. by @AsakusaRinne in #700
• Add LLaMA3 chat session example. by @AsakusaRinne in #708
• Improved Example Docs by @martindevans in #728

Bug Fixes

• fix: llava backend ignores avx and cuda. by @AsakusaRinne in #642
• Embeddings correction by @zsogitbe in #674
• Fix context params defaults by @dlyz in #729
• Made special tokens included in prompts tokenize as intended by @Lyrcaxis in #677
• Fixed Minor Issues With Model Loading by @martindevans in #697
• Empty batch check by @martindevans in #691
• Optimize Semantic Kernel LLamaSharpChatCompletion when running with StatefulExecutorBase models by @kidkych in #671
• SemanticKernel: Correcting non-standard way of working with PromptExecutionSettings by @zsogitbe in #689
• Fix cublas build action by @martindevans in #725

Other Changes

• build(deps): bump System.Text.Json from 8.0.2 to 8.0.3 by @dependabot in #636
• build(deps): bump Microsoft.AspNetCore.OpenApi from 8.0.2 to 8.0.3 by @dependabot in #638
• build(deps): bump Microsoft.Extensions.Logging.Abstractions from 8.0.0 to 8.0.1 by @dependabot in #637
• April 2024 Binary Update by @martindevans in #662
• Slightly Safer Quantize Params by @martindevans in #698
• Removed Unnecessary Constructor From Safe Handles by @martindevans in #696
• ci: add workflow to check the spellings. by @AsakusaRinne in #710
• ci: add a workflow to check code format. by @AsakusaRinne in #709
• Remove Conversation.Prompt(String) by @martindevans in #719
• ci: add benchmark test. by @AsakusaRinne in #720

New Contributors

• @Lyrcaxis made their first contribution in #656
• @Norne9 made their first contribution in #711
• @ksanman made their first contribution in #714
• @dlyz made their first contribution in #729
• @eltociear made their first contribution in #730

Full Changelog: 0.11.1...v0.12.0

0.11.1 - LLaVA support

🎏 Major Changes

• LLaVA Support by @SignalRT in #556, #563, #609
• Chat session state management by @eublefar in #560
• Classifier Free Guidance by @martindevans in #536
• March Binary Update by @martindevans in #565
• SetDllImportResolver based loading by @martindevans in #603

📖 Documentation

• The documentation has been improved and no longer outdated, see LLamaSharp Documentation.

🔧 Bug Fixes

• Added conditional compilation code to progress_callback (in LlamaModelParams struct) by @clovisribeiro in #593
• Memory Disposal Tests by @martindevans in #551
• Fixed Publish File paths by @martindevans in #561
• llama_decode lock by @martindevans in #595
• BatchedExecutor Fixed Forking by @martindevans in #621
• Fixed off by one error in LLamaBatch sampling position by @martindevans in #626
• [LLama.KernelMemory] Fixed System.ArgumentException: EmbeddingMode must be true & #617 by @ChengYen-Tang in #615
• fix: the missing of llava_shared library. by @AsakusaRinne in #633

📌 Other Changes

• Removed llama_eval() by @martindevans in #553
• ChatSession: improve exception message by @swharden in #523
• Improve "embeddings" example by @swharden in #525
• Add path to find llama.dll for MAUI by @evolcano in #631
• LLama.Examples: improve model path prompt by @swharden in #526
• NativeLibraryConfig.WithLogs() overload to set log level by @swharden in #529
• LLamaSharp.Examples: Document Q&A with local storage by @swharden in #532
• Used AnsiConsole in a few more places by @martindevans in #534
• ReadOnlySpan<float> in ISamplingPipeline by @martindevans in #538
• KernelMemory update with adding the use of already loaded model by @zsogitbe in #630
• Add Link To Blazor Demo by @alexhiggins732 in #539
• Removed Obsolete SamplingApi by @martindevans in #552
• update readme.md backends by @warquys in #587
• docs: update the example in readme. by @AsakusaRinne in #604
• Update Semantic Kernel & Kernel Memory Package by @xbotter in #612
• BatchedExecutor.Create() method by @martindevans in #613
• LLamaBatch Logit Tracking by @martindevans in #624

🙌 New Contributors

• @swharden made their first contribution in #523
• @alexhiggins732 made their first contribution in #539
• @clovisribeiro made their first contribution in #593
• @warquys made their first contribution in #587
• @eublefar made their first contribution in #560
• @ChengYen-Tang made their first contribution in #615
• @evolcano made their first contribution in #631

Full Changelog: v0.10.0...0.11.0

0.10.0 - Phi2

Major Changes

• Update binaries feb 2024 by @martindevans in #479
• Add CLBLAST native library to native libraries build by @jasoncouture in #468
• Introduced a new BatchedExecutor by @martindevans in #503
• Swapped StatelessExecutor to use llama_decode! by @martindevans in #445
• LLamaToken Struct by @martindevans in #404

Bug Fixes

• KernelMemory EmbeddingMode bug correction by @zsogitbe in #485
• Normalize Embeddings by @martindevans in #507
• StreamingTextDecoder Fix & Tests by @martindevans in #428
• Tokenizer Fixes For Issue 430 by @martindevans in #433

Other Changes

• Use llama instead of libllama in [DllImport] by @jasoncouture in #465
• Updated Examples by @vikramvee in #502
• Added new file types to quantisation by @martindevans in #495
• Smaller Unit Test Model by @martindevans in #496
• Using AddRange in LLamaEmbedder by @martindevans in #499
• Small KV Cache Handling Improvements by @martindevans in #500
• Added increment and decrement operators to LLamaPos by @martindevans in #501
• Swapped GetEmbeddings to llama_decode by @martindevans in #474
• kv_cache_instance_methods by @martindevans in #454
• Removed IModelParams and IContextParams setters. by @martindevans in #472
• Managed LLamaBatch by @martindevans in #442
• Check Model Path Exists by @martindevans in #437
• Model Metadata Loading Cleanup by @martindevans in #438
• Added a check for EOS token in LLamaStatelessExecutor by @martindevans in #434
• Update README.md by @Oceania2018 in #427
• Gpu layer count change by @Kaotic3 in #424
• Improved exceptions in IModelParams for unknown KV override types. by @martindevans in #416

New Contributors

• @Kaotic3 made their first contribution in #424
• @Oceania2018 made their first contribution in #427
• @jasoncouture made their first contribution in #465
• @zsogitbe made their first contribution in #485
• @vikramvee made their first contribution in #502

Full Changelog: 0.9.1...v0.10.0