Architecture

DeepFellow Server

DeepFellow Server is the gateway to access MCP servers, Tools, DeepFellow Infra, Plugins, and Vector Data Base (VDB). It also provides authorization and administration layers to the system. It is also used for user and access management, business logic and all the other application-related operations.

Your DeepFellow Server should have at least 1 CPU and 2GB RAM.

DeepFellow Architecture – self-hosted infra, DeepFellow Server, MCPs, tools, OpenAI API and client interfaces

DeepFellow Infra

DeepFellow Infra is a self-hosted infrastructure stack that holds the computing power for AI models to work. It hosts LLMs, embedding models, and other AI services such as rerankers, speech-to-text and text-to-speech models, and image-generation models. It also registers and hosts MCP servers.

Manage DeepFellow Infra through the Infra Web Panel.

Connecting Infras into a Mesh

Assume you want to connect two Infras in a child-parent relation.

Decide which of the two machines will be the parent and which will be the child.
On the parent Infra, reveal its DF_MESH_KEY value by running deepfellow infra info with the --secret flag. API keys are masked by default:

$ deepfellow infra info --secret    # on parent Infra

💡      Information about DeepFellow Infra:
        NAME: infra
        INFRA_URL: https://df-infra-node-1.com
        INFRA_MESH_URL: wss://df-infra-node-1.com
        INFRA_PORT: 8086
        INFRA_IMAGE: github.simplito.com:5050/df/deepfellow-infra:latest
        MESH_KEY: dfmesh_dbc8c1cf-c07a-4bba-bda1-d89829be37bb
        INFRA_API_KEY: dfinfra_c7a934fc-5e1f-41b2-b10e-57016c18d516
        INFRA_ADMIN_API_KEY: dfinfraadmin_99d7df55-107b-4327-aeef-54e187f6c5aa
        CONNECT_TO_MESH_URL: undefined
        CONNECT_TO_MESH_KEY: undefined
        INFRA_DOCKER_SUBNET: deepfellow-infra-net
        INFRA_COMPOSE_PREFIX: dfd834zh_
        INFRA_DOCKER_CONFIG: /home/mark/.docker/config.json
        INFRA_STORAGE_DIR: /home/mark/.deepfellow/infra/storage
        HUGGING_FACE_TOKEN: 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
        CIVITAI_TOKEN: 7fea22cc5605cf498066059415157828

The list of environmental variables for the parent Infra appears.

The values of DF_CONNECT_TO_MESH_KEY and DF_CONNECT_TO_MESH_URL are undefined. This means the parent Infra is not connected as a child to any other Infra.

On the child Infra, verify that DF_INFRA_URL is set to an address the parent can reach.

DF_INFRA_URL on the child must resolve from the parent's network. A local hostname such as http://infra:8086 works inside a Docker network but is unreachable from a parent on a different host (for example, one hosted on AWS). If the child is not publicly accessible, expose it through a public IP, a domain name, or a tunneling service such as ngrok, bore.pub, or localhost.run, then update the value:

deepfellow infra env set DF_INFRA_URL <var>PUBLIC_URL</var>

To connect the child Infra to the parent, type the following on the child:

deepfellow infra connect wss://df-infra-node-1.com dfmesh_dbc8c1cf-c07a-4bba-bda1-d89829be37bb

The child Infra calls the parent Infra with DF_MESH_KEY. If the parent Infra accepts the request, it connects the child Infra. Once connected, the child Infra has its DF_CONNECT_TO_MESH_KEY and DF_CONNECT_TO_MESH_URL values set:

$ deepfellow infra info --secret    # on child Infra
💡      Information about DeepFellow Infra:
        NAME: infra
        INFRA_URL: https://df-infra-node-2.com
        INFRA_MESH_URL: wss://df-infra-node-2.com
        INFRA_PORT: 8086
        INFRA_IMAGE: github.simplito.com:5050/df/deepfellow-infra:latest
        MESH_KEY: dfmesh_05f1b75e-9951-4dda-88b3-2868646d26c7
        INFRA_API_KEY: dfinfra_c7a934fc-5e1f-41b2-b10e-57016c18d516
        INFRA_ADMIN_API_KEY: dfinfraadmin_99d7df55-107b-4327-aeef-54e187f6c5aa
        CONNECT_TO_MESH_KEY: dfmesh_dbc8c1cf-c07a-4bba-bda1-d89829be37bb
        CONNECT_TO_MESH_URL: https://df-infra-node-1.com
        INFRA_DOCKER_SUBNET: deepfellow-infra-net
        INFRA_COMPOSE_PREFIX: dfd834zh_
        INFRA_DOCKER_CONFIG: /home/helen/.docker/config.json
        INFRA_STORAGE_DIR: /home/helen/.deepfellow/infra/storage
        HUGGING_FACE_TOKEN: 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
        CIVITAI_TOKEN: 7fea22cc5605cf498066059415157828

This means the child Infra is connected as a child to another Infra located at https://df-infra-node-1.com.

Repeat the process to build the topology of Infras you want.

To disconnect the child Infra and its descendants from the parent Infra, type the following on the child Infra:

deepfellow infra disconnect

DeepFellow example specification with sample hardware and its estimated costs

Example Infra Specification and Cost

DeepFellow Infra runs on self-hosted hardware or on a cloud GPU instance. Connect multiple Infra nodes into a Mesh to go from a single GPU to a multi-node cluster.

CardIcon