Magnitude

en inglés An open source inference server that makes local inference easy for agents by profiling your hardware, recommending compatible models, and running them tuned to your machine.