Skip to content

Commit 17dde75

Browse files
lunamidori5lunamidori5
authored
How To (Updates and Fixes) (mudler#1456)
* Update easy-setup-embeddings.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Update easy-setup-docker-cpu.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Update easy-setup-docker-gpu.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Update and rename easy-setup-docker-cpu.md to easy-setup-docker.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Update easy-setup-docker.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Update easy-setup-docker.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Update _index.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Update easy-setup-docker.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Update easy-setup-docker.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Delete docs/content/howtos/easy-setup-docker-gpu.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Update _index.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Update easy-setup-sd.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> * Update easy-setup-sd.md Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> --------- Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com>
1 parent 1fc3a37 commit 17dde75

5 files changed

Lines changed: 45 additions & 181 deletions

File tree

‎docs/content/howtos/_index.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,7 @@ weight = 9
88

99
This section includes LocalAI end-to-end examples, tutorial and how-tos curated by the community and maintained by [lunamidori5](https://github.com/lunamidori5).
1010

11-
- [Setup LocalAI with Docker on CPU]({{%relref "howtos/easy-setup-docker-cpu" %}})
12-
- [Setup LocalAI with Docker With CUDA]({{%relref "howtos/easy-setup-docker-gpu" %}})
11+
- [Setup LocalAI with Docker]({{%relref "howtos/easy-setup-docker" %}})
1312
- [Seting up a Model]({{%relref "howtos/easy-model" %}})
1413
- [Making Text / LLM requests to LocalAI]({{%relref "howtos/easy-request" %}})
1514
- [Making Photo / SD requests to LocalAI]({{%relref "howtos/easy-setup-sd" %}})

‎docs/content/howtos/easy-setup-docker-cpu.md‎

Lines changed: 0 additions & 137 deletions
This file was deleted.

docs/content/howtos/easy-setup-docker-gpu.md renamed to docs/content/howtos/easy-setup-docker.md

Lines changed: 39 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11

22
+++
33
disableToc = false
4-
title = "Easy Setup - GPU Docker"
4+
title = "Easy Setup - Docker"
55
weight = 2
66
+++
77

@@ -12,26 +12,13 @@ weight = 2
1212

1313
We are going to run `LocalAI` with `docker compose` for this set up.
1414

15-
Lets Setup our folders for ``LocalAI``
16-
{{< tabs >}}
17-
{{% tab name="Windows (Batch)" %}}
15+
Lets setup our folders for ``LocalAI`` (run these to make the folders for you if you wish)
1816
```batch
1917
mkdir "LocalAI"
2018
cd LocalAI
2119
mkdir "models"
2220
mkdir "images"
2321
```
24-
{{% /tab %}}
25-
26-
{{% tab name="Linux (Bash / WSL)" %}}
27-
```bash
28-
mkdir -p "LocalAI"
29-
cd LocalAI
30-
mkdir -p "models"
31-
mkdir -p "images"
32-
```
33-
{{% /tab %}}
34-
{{< /tabs >}}
3522

3623
At this point we want to set up our `.env` file, here is a copy for you to use if you wish, Make sure this is in the ``LocalAI`` folder.
3724

@@ -51,7 +38,7 @@ GALLERIES=[{"name":"model-gallery", "url":"github:go-skynet/model-gallery/index.
5138
MODELS_PATH=/models
5239

5340
## Enable debug mode
54-
# DEBUG=true
41+
DEBUG=true
5542

5643
## Disables COMPEL (Lets Stable Diffuser work, uncomment if you plan on using it)
5744
# COMPEL=0
@@ -84,28 +71,54 @@ BUILD_TYPE=cublas
8471

8572
Now that we have the `.env` set lets set up our `docker-compose` file.
8673
It will use a container from [quay.io](https://quay.io/repository/go-skynet/local-ai?tab=tags).
74+
75+
76+
{{< tabs >}}
77+
{{% tab name="CPU Only" %}}
78+
Also note this `docker-compose` file is for `CPU` only.
79+
80+
```docker
81+
version: '3.6'
82+
83+
services:
84+
api:
85+
image: quay.io/go-skynet/local-ai:{{< version >}}
86+
tty: true # enable colorized logs
87+
restart: always # should this be on-failure ?
88+
ports:
89+
- 8080:8080
90+
env_file:
91+
- .env
92+
volumes:
93+
- ./models:/models
94+
- ./images/:/tmp/generated/images/
95+
command: ["/usr/bin/local-ai" ]
96+
```
97+
{{% /tab %}}
98+
99+
{{% tab name="GPU and CPU" %}}
87100
Also note this `docker-compose` file is for `CUDA` only.
88101

89102
Please change the image to what you need.
90103
{{< tabs >}}
91104
{{% tab name="GPU Images CUDA 11" %}}
92105
- `master-cublas-cuda11`
93106
- `master-cublas-cuda11-core`
94-
- `v2.0.0-cublas-cuda11`
95-
- `v2.0.0-cublas-cuda11-core`
96-
- `v2.0.0-cublas-cuda11-ffmpeg`
97-
- `v2.0.0-cublas-cuda11-ffmpeg-core`
107+
- `{{< version >}}-cublas-cuda11`
108+
- `{{< version >}}-cublas-cuda11-core`
109+
- `{{< version >}}-cublas-cuda11-ffmpeg`
110+
- `{{< version >}}-cublas-cuda11-ffmpeg-core`
98111

99112
Core Images - Smaller images without predownload python dependencies
100113
{{% /tab %}}
101114

102115
{{% tab name="GPU Images CUDA 12" %}}
103116
- `master-cublas-cuda12`
104117
- `master-cublas-cuda12-core`
105-
- `v2.0.0-cublas-cuda12`
106-
- `v2.0.0-cublas-cuda12-core`
107-
- `v2.0.0-cublas-cuda12-ffmpeg`
108-
- `v2.0.0-cublas-cuda12-ffmpeg-core`
118+
- `{{< version >}}-cublas-cuda12`
119+
- `{{< version >}}-cublas-cuda12-core`
120+
- `{{< version >}}-cublas-cuda12-ffmpeg`
121+
- `{{< version >}}-cublas-cuda12-ffmpeg-core`
109122

110123
Core Images - Smaller images without predownload python dependencies
111124
{{% /tab %}}
@@ -135,6 +148,8 @@ services:
135148
- ./images/:/tmp/generated/images/
136149
command: ["/usr/bin/local-ai" ]
137150
```
151+
{{% /tab %}}
152+
{{< /tabs >}}
138153

139154

140155
Make sure to save that in the root of the `LocalAI` folder. Then lets spin up the Docker run this in a `CMD` or `BASH`

‎docs/content/howtos/easy-setup-embeddings.md‎

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -12,25 +12,14 @@ curl http://localhost:8080/models/apply -H "Content-Type: application/json" -d '
1212
}'
1313
```
1414

15-
Now we need to make a ``bert.yaml`` in the models folder
16-
```yaml
17-
backend: bert-embeddings
18-
embeddings: true
19-
name: text-embedding-ada-002
20-
parameters:
21-
model: bert
22-
```
23-
24-
**Restart LocalAI after you change a yaml file**
25-
2615
When you would like to request the model from CLI you can do
2716

2817
```bash
2918
curl http://localhost:8080/v1/embeddings \
3019
-H "Content-Type: application/json" \
3120
-d '{
3221
"input": "The food was delicious and the waiter...",
33-
"model": "text-embedding-ada-002"
22+
"model": "bert-embeddings"
3423
}'
3524
```
3625

‎docs/content/howtos/easy-setup-sd.md‎

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ weight = 2
55
+++
66

77
To set up a Stable Diffusion model is super easy.
8-
In your models folder make a file called ``stablediffusion.yaml``, then edit that file with the following. (You can change ``Linaqruf/animagine-xl`` with what ever ``sd-lx`` model you would like.
8+
In your ``models`` folder make a file called ``stablediffusion.yaml``, then edit that file with the following. (You can change ``Linaqruf/animagine-xl`` with what ever ``sd-lx`` model you would like.
99
```yaml
1010
name: animagine-xl
1111
parameters:
@@ -21,8 +21,7 @@ diffusers:
2121
2222
If you are using docker, you will need to run in the localai folder with the ``docker-compose.yaml`` file in it
2323
```bash
24-
docker-compose down #windows
25-
docker compose down #linux/mac
24+
docker compose down
2625
```
2726

2827
Then in your ``.env`` file uncomment this line.
@@ -32,14 +31,13 @@ COMPEL=0
3231

3332
After that we can reinstall the LocalAI docker VM by running in the localai folder with the ``docker-compose.yaml`` file in it
3433
```bash
35-
docker-compose up #windows
36-
docker compose up #linux/mac
34+
docker compose up -d
3735
```
3836

3937
Then to download and setup the model, Just send in a normal ``OpenAI`` request! LocalAI will do the rest!
4038
```bash
4139
curl http://localhost:8080/v1/images/generations -H "Content-Type: application/json" -d '{
4240
"prompt": "Two Boxes, 1blue, 1red",
43-
"size": "256x256"
41+
"size": "1024x1024"
4442
}'
4543
```

0 commit comments

Comments
 (0)