Click here to Skip to main content
15,569,845 members
Articles / Artificial Intelligence / Tensorflow
Article
Posted 16 Jan 2022

Stats

367.9K views
110.3K downloads
107 bookmarked

CodeProject.AI Server: AI the easy way.

Rate me:
Please Sign up or sign in to vote.
5.00/5 (47 votes)
27 Jan 20239 min read
Version 2.0.7. Our fast, free, self-hosted Artificial Intelligence Server for any platform, any language
CodeProject.AI Server is a locally installed, self-hosted, fast, free and Open Source Artificial Intelligence server for any platform, any language. No off-device or out of network data transfer, no messing around with dependencies, and able to be used from any platform, any language. Runs as a Windows Service or a Docker container.

Image 1

Downloads

Utility Scripts 

CodeProject.AI Server: An Artificial Intelligence server

Think of CodeProject.AI server like a database server: you install it, ir runs in the background, and provides AI operations for any application via a simple API. 

CodeProject.AI server runs as a Windows service or under Docker. Any language that can make HTTP calls can access the service, and the server does not require an external internet connection. Your data stays in your network.

What's New - 2.0

  • 2.0.7: Improved logging, both file based and in the dashboard, module installer/uninstaller bug fixes, 
  • 2.0.6: Corrected issues with downloadable modules installer
  • Our new Module Registry: download and install modules at runtime via the dashboard
  • Improved performance for the Object Detection modules
  • Optional YOLO 3.1 Object Detection module for older GPUs
  • Optimised RAM use
  • Support for Raspberry Pi 4+. Code and run natively directly on the Raspberry Pi using VSCode natively
  • Revamped dashboard
  • New timing reporting for each API call
  • New, simplified setup and install scripts

Please see our CUDA Notes for information on setting up, and restrictions around, Nvidia cards and CUDA support.

If you are upgrading: when the dashboard launches it might be necessary to force-reload (Ctrl+R on Windows) the dashboard to ensure you are viewing the latest version

Why we built CodeProject.AI Server

  • AI programming is something every single developer should be aware of

    We wanted a fun project we could use to help teach developers and get them involved in AI. We'll be using CodeProject.AI Server as a focus for articles and exploration to make it fun and painless to learn AI programming

    We want your contributions!
  • AI coding examples have too many moving parts

    You need to install packages and languages and extensions to tools, and then updates and libraries (but version X, not version Y) and then you have to configure paths and...Oh, you want to run on Windows not Linux? In that case you need to... It's all too hard. There was much yelling at CodeProject.

    CodeProject.AI Server includes everything you need in a single installer. CodeProject.AI Server also provides an installation script that will setup your dev environment and get you debugging within a couple of clicks.
  • AI solutions often require the use of cloud services

    If you trust the cloud provider, or understand the billing structure, or can be assured you aren't sending sensitive data or won't go over the free tier this is fine. If you have a webcam inside your house, or can't work out how much AWS will charge, it's not so OK

    CodeProject.AI Server can be installed locally. Your machine, your network, no data needs to leave your device.

Supported Platforms

Image 2 Image 3 Image 4 Image 5 Image 6 Image 7 Image 8 Image 9
Windows macOS macOS-arm64 Ubuntu Raspberry Pi Docker Visual Studio
2019+
Visual Studio
Code

Where to start: Quick links

  1. Download and install in Windows or Docker. Fire up the dashboard at http://localhost:32168.
  2. Read the docs (or at least browse through to get a feel)
  3. Learn how to configure the modules if necessary
  4. Download, build and run the code

Quick Introductions

1: Running and playing with the features

  1. Install and Run
    1. For a Windows Service, Download the latest version, install, and launch the shortcut to the server's dashboard on your desktop or open a browser to http://localhost:32168.

      If you wish to take advantage of a CUDA enabled NVidia GPU, please ensure you have the CUDA drivers installed before you install CodeProject.AI
       
    2. For a Docker Container for 64 Bit Linux run
      docker run -p 32168:32168 --name CodeProject.AI-Server -d -v <local directory>:/etc/codeproject/ai codeproject/ai-server
      where <local directory> is some existing directory on the host machine such as
      1. C:\ProgramData\CodeProject/AI on Windows
      2. /usr/share/CodeProject/AI on Linux
    3. For Docker GPU (nVidia CUDA), please use 
      docker run --gpus all -p 32168:32168 --name CodeProject.AI-Server -d -v <local directory>:/etc/codeproject/ai codeproject/ai-server:gpu
  2. On the dashboard, at the bottom, is a link to the demo playground. Open that and play!

2: Running and debugging the code

  1. Clone the CodeProject CodeProject.AI Server repository.
  2. Make sure you have Visual Studio Code or Visual Studio 2019+ installed.
  3. Run the setup script in /Installers/Dev
  4. Debug the front-end server application (see notes below, but it's easy)

3. Using CodeProject.AI Server in my application

Here's an example of using the API for scene detection using a simple JavaScript call:

HTML
<html>
<body>
Detect the scene in this file: <input id="image" type="file" />
<input type="button" value="Detect Scene" onclick="detectScene(image)" />

<script>
function detectScene(fileChooser) {
    var formData = new FormData();
    formData.append('image', fileChooser.files[0]);

    fetch('http://localhost:5000/v1/vision/detect/scene', {
        method: "POST",
        body: formData
    })
    .then(response => {
        if (response.ok) response.json().then(data => {
            console.log(`Scene is ${data.label}, ${data.confidence} confidence`)
        });
    });
}
</script>
</body>
</html>

You can include the CodeProject.AI Server installer (or just a link to the latest version of the installer) in your own apps and installers and voila, you have an AI enabled app.

See the API documentation for a complete rundown of functionality

Notes on CUDA and Nvidia support

If you have a CUDA enabled Nvidia card please then ensure you

  1. install the CUDA Drivers 
  2. Install CUDA Toolkit 11.7.
  3. Download and run our cuDNN install script to install cuDNN.

Nvidia downloads and drivers are challenging! Please ensure you download a driver that is compatible with CUDA 11.7, which generally means the CUDA driver version 516.94 or below. Version 522.x or above may not work. You may need to refer to the release notes for each driver to confirm

Since we are using CUDA 11.7 (which has support for compute capability 3.7 and above).we can only support NVidia CUDA cards that are equal to or better than a GK210 or Tesla K80 card. Please refer to this table of supported cards to determine if your card has compute capability 3.7 or above.

Newer cards such as the GTX 10xx, 20xx and 30xx series, RTX, MX series are fully supported.

AI is a memory intensive operation. Some cards with 2GB RAM or less may struggle in some situations. Using the dashboard you can either disable modules you don't need, or disable GPU support entirely for one or more modules. This will free up memory and help get you back on track.

What does it include?

CodeProject.AI Server includes

  • A HTTP REST API Server. The server listens for requests from other apps, passes them to the backend analysis services for processing, and then passes the results back to the caller. It runs as a simple self-contained web service on your device.
  • Backend Analysis services. The brains of the operation is in the analysis services sitting behind the front end API. All processing of data is done on the current machine. No calls to the cloud and no data leaving the device.
  • The Source Code, naturally.

What can it do?

CodeProject.AI Server can currently

  • Detect objects in images
  • Detect faces in images
  • Detect the type of scene represented in an image
  • Recognise faces that have been registered with the service
  • Perform detection on custom models

The development environment also provides modules that can 

  • Remove a background from an image
  • Blur a background from an image
  • Enhance the resolution of an image
  • Pull out the most important sentences in text to generate a text summary
  • Prove sentiment analysis on text

We will be constantly expanding the feature list.

Our Goals

  • To promote AI development and inspire the AI developer community to dive in and have a go. Artificial Intelligence is a huge paradigm change in the industry and all developers owe it to themselves to experiment in and familiarize themselves with the technology. CodeProject.AI Server was built as a learning tool, a demonstration, and a library and service that can be used out of the box.
  • To make AI development easy. It's not that AI development is that hard. It's that there are so, so many options. Our architecture is designed to allow any AI implementation to find a home in our system, and for our service to be callable from any language.
  • To focus on core use-cases. We're deliberately not a solution for everyone. Instead we're a solution for common day-to-day needs. We will be adding dozens of modules and scores of AI capabilities to our system, but our goal is always clarity and simplicity over a 100% solution.
  • To tap the expertise of the Developer Community. We're not experts but we know a developer or two out there who are. The true power of CodeProject.AI Server comes from the contributions and improvements from our AI community.

How to Guides

License

CodeProject.AI Server is licensed under the Server-Side Public License

Release Notes and Roadmap

Coming up

  • More modules.
  • Chaining modules.

Previous versions

Release 1.6.x Beta

  • Optimised RAM use
  • Ability to enable / disable modules and GPU support via the dashboard
  • REST settings API for updating settings on the fly
  • Apple M1/M2 GPU support
  • Workarounds for some Nvidia cards
  • Async processes and logging for a performance boost
  • Breaking: the CustomObjectDetection is now part of ObjectDetectionYolo
     
  • Performance fix for CPU + video demo.
  • Patch 1.6.7: potential memory leak addressed
  • Patch 1.6.8: image handling improvements on Linux, multi-thread ONNX on .NET

Release 1.5.6.2 Beta

  • Docker nVidia GPU support
  • Further performance improvements
  • cuDNN install script to help with nVidia driver and toolkit installation
  • Bug fixes

Release 1.5.6 Beta

  • nVidia GPU support for Windows
  • Perf improvements to Python modules
  • Work on the Python SDK to make creating modules easier
  • Dev installers now drastically simplified for those creating new modules
  • Added SuperResolution as a demo module

Release 1.5 Beta

  • Support for custom models

Release 1.3.x Beta

  • Refactored and improved setup and module addition system
  • Introduction of modulesettings.json files
  • New analysis modules

Release 1.2.x Beta

  • Support for Apple Silicon for development mode
  • Native Windows installer
  • Runs as Windows Service
  • Run in a Docker Container
  • Installs and Builds using VSCode in Linux (Ubuntu), macOS and Windows, as well as Visual Studio on Windows
  • General optimisation of the download payload sizes

Previous

  • We started with a proof of concept on Windows 10+ only. Installs we via a simple BAT script, and the code has is full of exciting sharp edges. A simple dashboard and playground are included. Analysis is currently Python code only
  • Version checks are enabled to alert users to new versions
  • A new .NET implementation scene detection using the YOLO model to ensure the codebase is platform and tech stack agnostic
  • Blue Iris integration completed

Written By
Software Developer CodeProject Solutions
Canada Canada
The CodeProject team have been writing software, building communities, and hosting CodeProject.com for over 20 years. We are passionate about helping developers share knowledge, learn new skills, and connect. We believe everyone can code, and every contribution, no matter how small, helps.

The CodeProject team is currently focussing on CodeProject.AI Server, a stand-alone, self-hosted server that provides AI inferencing services on any platform for any language. Learn AI by jumping in the deep end with us: codeproject.com/AI.
This is a Organisation

4 members

Comments and Discussions

 
PinnedA note on custom models and Blue Iris PinPopular
Chris Maunder24-Sep-22 12:18
cofounderChris Maunder24-Sep-22 12:18 
If you are a Blue Iris user and you are using custom models, then you would notice that the option, in Blue Iris, to set the custom model location is greyed out. This is because Blue Iris does not currently make changes to CodeProject.AI Server's settings. It can be done by manually starting CodeProject.AI with command line parameters (not a great solution), or editing the module settings files (a little messy), or setting system-wide environment variables (way easier). For version 1.6 we added an API to allow any app to change our settings programmatically, and we take care of stopping/restarting things and persisting the changes.

So: Blue Iris doesn't currently change CodeProject.AI Server's settings, so it doesn't provide you a way to change the custom model folder location from within Blue Iris.

Blue Iris will still use the contents of this folder to determine the calls it makes. If you don't specify a model to use in the Custom Models textbox, then Blue Iris will use all models in the custom models folder that it knows about.



Here we've specified a specific model to use. The Blue Iris help file explains more about how this works, including inclusive and exclusive filters on the models it finds.

CodeProject.AI Server doesn't know about Blue Iris' folder, so it can't tell what models it may be expected to use, nor can it tell Blue Iris about what models CodeProject.AI server has available. Our API allows Blue Iris to get a list of the AI models installed with CodeProject.AI Server, and also to set the folder where these models reside. But Blue Iris doesn't, yet, use that API.

So we do a hack.

At install time we sniff the registry to find where Blue Iris thinks the custom models should be. We then make empty copies of the models that we have, and copy them into that folder. If the folder doesn't exist (eg you were using C:\Program Files\CodeProject\AI\AnalysisLayer\CustomObjectDetection\assets, which no longer exists) then we create that folder, and then copy over the empty files.

When Blue Iris looks in that folder to decide what custom calls it can make, it sees the models, notes their names, and uses those names in the calls. CodeProject.AI Server has those models, so when the calls come through we can process them.

Blue Iris doesn't use the models. It uses the list of model names.

If you have your own models in the Blue Iris folder

You will need to copy them to the CodeProject.AI server's custom model folder (by default this is C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\custom-models)

If you've modified the registry and have your own custom models

If you were using a folder in C:\Program Files\CodeProject\AI\AnalysisLayer\CustomObjectDetection\ (which no longer existed after the upgrade, but was recreated by our hack) you'll need to re-copy your custom model into that folder.

The simplest solutions are:
  1. Modify the registry (Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Perspective Software\Blue Iris\Options\AI, key 'deepstack_custompath') so Blue Iris looks in C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\custom-models for custom models, and copy your models into there.

    or

  2. Modify C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\modulesettings.json file and set CUSTOM_MODELS_DIR to be whatever Blue Iris thinks the custom model folder is.
cheers
Chris Maunder

PinnedFound a bug? Got a Suggestion? Please read first! Pin
Chris Maunder23-Sep-22 13:51
cofounderChris Maunder23-Sep-22 13:51 
QuestionMemory Leaks with 2.0.7 Pin
StefanLU11hrs 51mins ago
StefanLU11hrs 51mins ago 
AnswerRe: Memory Leaks with 2.0.7 Pin
laupid11hrs 23mins ago
laupid11hrs 23mins ago 
GeneralRe: Memory Leaks with 2.0.7 Pin
Chris Maunder2hrs 49mins ago
cofounderChris Maunder2hrs 49mins ago 
QuestionCode project unhandled exception failed to load configuration file Pin
Member 1591767419hrs 54mins ago
Member 1591767419hrs 54mins ago 
AnswerRe: Code project unhandled exception failed to load configuration file Pin
Chris Maunder2hrs 48mins ago
cofounderChris Maunder2hrs 48mins ago 
QuestionHow to intercept or get JSON of predictions results? Pin
Member 159169887-Feb-23 6:43
professionalMember 159169887-Feb-23 6:43 
AnswerRe: How to intercept or get JSON of predictions results? Pin
Chris Maunder2hrs 46mins ago
cofounderChris Maunder2hrs 46mins ago 
GeneralRe: How to intercept or get JSON of predictions results? Pin
Mike Lud2hrs 11mins ago
communityengineerMike Lud2hrs 11mins ago 
GeneralMy vote of 5 Pin
Member 158762416-Feb-23 8:02
Member 158762416-Feb-23 8:02 
QuestionHttpClient.Timeout error in Docker Pin
Turbo Diesel Operator [W5TDO]5-Feb-23 18:34
Turbo Diesel Operator [W5TDO]5-Feb-23 18:34 
AnswerRe: HttpClient.Timeout error in Docker Pin
Chris Maunder2hrs 45mins ago
cofounderChris Maunder2hrs 45mins ago 
QuestionNvidia T400 2GB Model Size - Does the setting in BI work? Pin
coolspot185-Feb-23 9:19
coolspot185-Feb-23 9:19 
AnswerRe: Nvidia T400 2GB Model Size - Does the setting in BI work? Pin
Member 157448557-Feb-23 13:43
Member 157448557-Feb-23 13:43 
GeneralRe: Nvidia T400 2GB Model Size - Does the setting in BI work? Pin
Chris Maunder2hrs 43mins ago
cofounderChris Maunder2hrs 43mins ago 
QuestionCPAI 1.6.8-Beta General Exception "Tensor Error" Pin
MrMike20004-Feb-23 20:42
MrMike20004-Feb-23 20:42 
AnswerRe: CPAI 1.6.8-Beta General Exception "Tensor Error" Pin
Chris Maunder2hrs 41mins ago
cofounderChris Maunder2hrs 41mins ago 
QuestionModel Input Resolution Pin
jnsears4-Feb-23 11:08
jnsears4-Feb-23 11:08 
Questionv2.0.2 is the most stable version for me Pin
Techno Vol3-Feb-23 4:34
Techno Vol3-Feb-23 4:34 
AnswerRe: v2.0.2 is the most stable version for me Pin
Member 111023393-Feb-23 5:22
Member 111023393-Feb-23 5:22 
GeneralRe: v2.0.2 is the most stable version for me Pin
Techno Vol3-Feb-23 6:24
Techno Vol3-Feb-23 6:24 
AnswerRe: v2.0.2 is the most stable version for me Pin
Chris Maunder3-Feb-23 7:43
cofounderChris Maunder3-Feb-23 7:43 
GeneralRe: v2.0.2 is the most stable version for me Pin
Techno Vol3-Feb-23 8:31
Techno Vol3-Feb-23 8:31 
GeneralRe: v2.0.2 is the most stable version for me Pin
Graeme Orr3-Feb-23 13:30
Graeme Orr3-Feb-23 13:30 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Flags: AnsweredNeed more info

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.