
.. _`quickstart-server`:

Quickstart: permanent install on server/laptop
==========================================================

.. include:: links.rst

This document walks you through setting up your own ``devpi-server``
instance, controlling it with supervisor_ on a UNIX-like system or
launchd_ on Mac OS X and (optionally) serving it through nginx_ on a
UNIX-like system. It also shows how to create a first user and index.

.. warning::

    If you intend to expose your server to the internet, you should read
    about :doc:`securing it properly <adminman/security>`.

Installing devpi-server
-----------------------

Install or upgrade ``devpi-server``::

    pip install -U devpi-server
    # if you want the web interface
    pip install -U devpi-web


And let's check the version::

    $ devpi-server --version
    6.17.0

.. _genconfig:


Using a configuration file for devpi-server
-------------------------------------------

A `strict YAML`_ conform configuration file can be used in place of the command line options.

A configuration file named ``devpi-server.yml`` will be looked for in your systems and your users default configuration location using the `platformdirs`_ library.

.. note::

    For ``platformdirs`` the application name is ``devpi-server``, and the application author is ``devpi``.
    Thus, for example, on Linux the user configuration file would be ``~/.config/devpi-server/devpi-server.yml``.

With the `-c/--configfile` option the location can be provided explicitly.

The configuration file must start with a mapping and all devpi-server settings must be placed below the `devpi-server` key.
For example:

.. code-block:: yaml

    devpi-server:
      serverdir: /var/db/devpi-server
      secretfile: /etc/devpi-server/secret
      host: localhost
      port: 8080

.. _`strict YAML`: https://pypi.org/project/strictyaml/
.. _`platformdirs`: https://pypi.org/project/platformdirs/


Using environment variables for devpi-server configuration
----------------------------------------------------------

Environment variables can now be used for configuration.

The command line option name needs to be changed to uppercase, prefixed with DEVPISERVER\_ and dashes replaced by underscores.
For example ``--restrict-modify`` becomes ``DEVPISERVER_RESTRICT_MODIFY``.


Priority of configuration options
---------------------------------

Values from configuration files have the lowest priority.
Next are the environment variables.
The command line options have the highest priority and overwrite all of the above.


generating example config files for supervisor/nginx/cron/launchd
-----------------------------------------------------------------

Use ``devpi-gen-config`` to write out several example config files
into the ``gen-config`` directory (relative to where you invoke it
from).  The files come with some defaults to integrate with your own
permanent hosting infrastructure.

If you have improvements or additions to the example configs please
share them with a
`Merge/Pull Request on the configuration template files <https://github.com/devpi/devpi/tree/main/server/devpi_server/cfg>`_ and
`devpi-gen-config implementation <https://github.com/devpi/devpi/blob/main/server/devpi_server/genconfig.py>`_ along with a little doc section.

Here is an example run::

    $ devpi-gen-config --port 4040 --serverdir ~/mydevpiserver
    It is highly recommended to use a configuration file for devpi-server, see --configfile option.
    wrote gen-config/crontab
    wrote gen-config/net.devpi.plist
    wrote gen-config/launchd-macos.txt
    wrote gen-config/nginx-devpi.conf
    wrote gen-config/nginx-devpi-caching.conf
    wrote gen-config/supervisor-devpi.conf
    wrote gen-config/supervisord.conf
    wrote gen-config/devpi.service
    wrote gen-config/windows-service.txt

Below we look at the config files in a bit of detail.


.. _`configured nginx`:

``nginx-devpi.conf``: nginx site config
++++++++++++++++++++++++++++++++++++++++++++++++++

The nginx site configuration as generated by the above example run::

    $ cat gen-config/nginx-devpi.conf
    map $http_x_forwarded_proto $x_scheme {
        default $scheme;
        http http;
        https https;
    }

    server {
        server_name localhost $hostname "";
        listen 80;
        gzip             on;
        gzip_min_length  2000;
        gzip_proxied     any;
        # add application/vnd.pypi.simple.v1+json to the gzip_types
        gzip_types  text/plain text/css text/xml
                    application/json application/vnd.pypi.simple.v1+json
                    application/javascript text/javascript
                    application/xml application/xml+rss;

        proxy_read_timeout 60s;
        client_max_body_size 64M;

        # set to where your devpi-server state is on the filesystem
        root /tmp/home/mydevpiserver;

        # try serving static files directly
        location ~ /\+f/ {
            # workaround to pass non-GET/HEAD requests through to the named location below
            error_page 418 = @proxy_to_app;
            if ($request_method !~ (GET)|(HEAD)) {
                return 418;
            }

            expires max;
            try_files /+files$uri @proxy_to_app;
        }

        # try serving docs directly
        location ~ /\+doc/ {
            # if the --documentation-path option of devpi-web is used,
            # then the root must be set accordingly here
            root /tmp/home/mydevpiserver;
            try_files $uri @proxy_to_app;
        }

        location / {
            # workaround to pass all requests to / through to the named location below
            error_page 418 = @proxy_to_app;
            return 418;
        }

        location @proxy_to_app {
            proxy_pass http://localhost:4040;
            # the $x_scheme variable is only required if nginx is behind another
            # proxy (often the case in container environments),
            # if your nginx is the only proxy server, the $scheme variable can be
            # used and the map $http_x_forwarded_proto $x_scheme above be removed
            proxy_set_header X-Forwarded-Proto $x_scheme;
            proxy_set_header X-outside-url $x_scheme://$http_host;
            proxy_set_header X-Real-IP $remote_addr;
        }
    }

The devpi specific settings are found in the location directives
where we first try to serve static files through nginx before
deferring to the devpi-server server instance.  The X-outside URL
settings drive the devpi-server to generate correct URLs 
because it might otherwise assume it is exposed at ``http://localhost:4040``
and generate URLs accordingly.


supervisor site configuration
+++++++++++++++++++++++++++++++++++++++++++++++

Let's look at the supervisor configuration files.  The 
``supervisor-devpi.conf`` one presumes you have a main
supervisor configuration on your system and
and just drop the include config file into a directory like
``/etc/supervisor/conf.d/`` on Ubuntu 12.04.  Here are the contents::

    $ cat gen-config/supervisor-devpi.conf
    [program:devpi-server]
    command=/home/devpi/devpi/bin/devpi-server --port 4040 --serverdir /tmp/home/mydevpiserver
    priority = 999
    startsecs = 5
    redirect_stderr = True
    autostart = True

If supervisord is running on your system you may, after copying
the config file to the supervisor ``conf.d`` directory, reload supervisord:

configuration directory 


crontab / start at bootup 
+++++++++++++++++++++++++

If you want to have things running at system startup and you are using
a standard cron, a modified copy of your user crontab has been amended
which you may inspect::

    $ cat gen-config/crontab
    # as an example you can use the @reboot special target to start the
    # http://supervisord.org process manager to manage devpi-server:
    # @reboot /usr/local/sbin/supervisord -c /home/testuser/etc/supervisor-devpi.conf

and install with::

    crontab TARGETDIR/etc/crontab


launchd / start at bootup (on Mac OS X)
+++++++++++++++++++++++++++++++++++++++

Although cron can be used on Mac OS X, the preferred service manager
on Mac OS X is launchd_. With launchd, you will be able to use the
``launchctl`` command to start, stop, and monitor your server like any
other Mac OS X service. An example property list file for your launch
agent has been generated for you::

    $ cat gen-config/net.devpi.plist
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
    	<key>Label</key>
    	<string>net.devpi</string>
    	<key>ProgramArguments</key>
    	<array>
    		<string>/home/devpi/devpi/bin/devpi-server</string>
    		<string>--port</string>
    		<string>4040</string>
    		<string>--serverdir</string>
    		<string>/tmp/home/mydevpiserver</string>
    	</array>
    	<key>RunAtLoad</key>
    	<true/>
    </dict>
    </plist>

You can install this agent for your user account (recommended) by running::

    mkdir -p ~/Library/LaunchAgents
    cp gen-config/net.devpi.plist ~/Library/LaunchAgents

You can also install to any of the system locations listed in the FILES section of :manpage:`launchd(8)`. However, this isn't recommended unless you know what you're doing.

devpi should now launch upon the next reboot. You can tell launchd to launch it right now by running the following::

    launchctl load ~/Library/LaunchAgents/net.devpi.plist
    launchctl start net.devpi

The argument passed to ``launchctl start`` corresponds to the ``Label`` keyword in the plist file, but also matches the file name by convention.

To manually stop the devpi server started by launchd::

    launchctl stop net.devpi

Please see the man pages :manpage:`launchd(8)`,
:manpage:`launchd.plist(5)`, and :manpage:`launchctl(1)` on your
system for more information on launchd.


Installing devpi server and client
---------------------------------------------

..
    $ rm -rf ~/.devpi/server

When started afresh, ``devpi-server`` will not contain any users
or indexes except for the root user and the ``root/pypi`` index
(see :ref:`using root/pypi index <install_first>`) which represents
and caches https://pypi.org packages.  Let's start a server
for the purposes of this tutorial in the background.

First initialize it::

    $ devpi-init
    INFO  NOCTX Loading node info from /tmp/home/.devpi/server/.nodeinfo
    INFO  NOCTX generated uuid: 446e22e0db5e41a5989fd671e98ec30b
    INFO  NOCTX wrote nodeinfo to: /tmp/home/.devpi/server/.nodeinfo
    INFO  NOCTX DB: Creating schema
    INFO  [Wtx-1] setting password for user 'root'
    INFO  [Wtx-1] created user 'root'
    INFO  [Wtx-1] created root user
    INFO  [Wtx-1] created root/pypi index
    INFO  [Wtx-1] fswriter0: committed at 0

Next we create configuration files for various process managers.
Note that we set the port we want to use::

    $ devpi-gen-config --port 4040
    It is highly recommended to use a configuration file for devpi-server, see --configfile option.
    wrote gen-config/crontab
    wrote gen-config/net.devpi.plist
    wrote gen-config/launchd-macos.txt
    wrote gen-config/nginx-devpi.conf
    wrote gen-config/nginx-devpi-caching.conf
    wrote gen-config/supervisor-devpi.conf
    wrote gen-config/supervisord.conf
    wrote gen-config/devpi.service
    wrote gen-config/windows-service.txt

Now for this tutorial we use supervisor_ to run the server.
For that we first need to create a supervisor.conf which includes the
generated configuration. For this tutorial it looks like this:

.. code-block:: ini

    [supervisord]

    [unix_http_server]
    file=%(here)s/supervisor.sock

    [supervisorctl]
    serverurl=unix://%(here)s/supervisor.sock

    [rpcinterface:supervisor]
    supervisor.rpcinterface_factory=supervisor.rpcinterface:make_main_rpcinterface

    [include]
    files = %(here)s/gen-config/supervisor-devpi.conf

Now we can start supervisor which will manage our server::

    $ supervisord -c gen-config/supervisord.conf

..
    $ waitforports -t 60 4040
    Waiting for 127.0.0.1:4040

The server will listen on ``http://localhost:4040`` and also serve
the devpi web interface if you have ``devpi-web`` installed.

In order to manage users and indices on our fresh server let's also
install the ``devpi-client`` package::

    $ pip install -U --pre -q devpi-client

You can install this client software on different hosts if you `configured nginx`_.

.. _auth:

Connecting to the server
++++++++++++++++++++++++++++++++++

If you `configured nginx`_, you can use the ``server_name``
variable content for connecting to the server, instead of the localhost.
For purposes of this tutorial, we use the URL
``http://localhost:4040`` of the background tutorial server we started above::

    $ devpi use http://localhost:4040
    using server: http://localhost:4040/ (not logged in)
    no current index: type 'devpi use -l' to discover indices
    venv for install/set commands: /tmp/docenv
    only setting venv pip/uv config, no global configuration changed
    /tmp/docenv/pip.conf: no config file exists
    /tmp/docenv/uv.toml: no config file exists
    always-set-cfg: no

At this point we are not connected to any index, just to the
root server.   And we are not logged in.

setting the root password
++++++++++++++++++++++++++++++++++

The first thing to do is to set a password for the ``root`` user.
For that we first need to login::

    $ devpi login root --password ''
    logged in 'root', credentials valid for 10.00 hours

and can then change it::

    $ devpi user -m root password=123
    /root changing password: ********
    user modified: root

Let's verify we don't have any other users::

    $ devpi user -l
    root

The root user can modify any index and any user configuration.
As we don't plan to work further with the root user, we can log off::

    $ devpi logoff
    login information deleted

Registering a new user
++++++++++++++++++++++++++++++++

Let's register ourselves a new regular non-root user::

    $ devpi user -c alice password=456  email=alice@example.com
    user created: alice

and then login::

    $ devpi login alice --password=456
    logged in 'alice', credentials valid for 10.00 hours

Alice can now create her new ``dev`` index::

    $ devpi index -c dev bases=root/pypi
    http://localhost:4040/alice/dev?no_projects=:
      type=stage
      bases=root/pypi
      volatile=True
      acl_upload=alice
      acl_toxresult_upload=:ANONYMOUS:
      mirror_whitelist=
      mirror_whitelist_inheritance=intersection

and use it ::

    $ devpi use alice/dev
    current devpi index: http://localhost:4040/alice/dev (logged in as alice)
    supported features: push-no-docs, push-only-docs, push-register-project, server-keyvalue-parsing
    venv for install/set commands: /tmp/docenv
    only setting venv pip/uv config, no global configuration changed
    /tmp/docenv/pip.conf: no config file exists
    /tmp/docenv/uv.toml: no config file exists
    always-set-cfg: no

Our ``alice/dev`` index derives from ``root/pypi``
which makes all pypi.org releases available.


automatically setting pip/easy_install config files
++++++++++++++++++++++++++++++++++++++++++++++++++++

You can cause devpi to set ``$HOME`` configuration files which will
cause ``pip`` and ``easy_install`` to use our in-use index server::

    $ devpi use --set-cfg alice/dev
    current devpi index: http://localhost:4040/alice/dev (logged in as alice)
    supported features: push-no-docs, push-only-docs, push-register-project, server-keyvalue-parsing
    venv for install/set commands: /tmp/docenv
    only setting venv pip/uv config, no global configuration changed
    /tmp/docenv/pip.conf: http://localhost:4040/alice/dev/+simple/
    /tmp/docenv/uv.toml: http://localhost:4040/alice/dev/+simple/
    always-set-cfg: no

This will modify or create common configuration files in your home directory
so that subsequent ``pip`` or ``easy_install`` invocations will work against
the ``user/indexname`` index.   You can configure ``devpi`` to perform
this configuration modification::

    $ devpi use --always-set-cfg=yes
    current devpi index: http://localhost:4040/alice/dev (logged in as alice)
    supported features: push-no-docs, push-only-docs, push-register-project, server-keyvalue-parsing
    venv for install/set commands: /tmp/docenv
    only setting venv pip/uv config, no global configuration changed
    /tmp/docenv/pip.conf: http://localhost:4040/alice/dev/+simple/
    /tmp/docenv/uv.toml: http://localhost:4040/alice/dev/+simple/
    always-set-cfg: yes

This will imply ``--set-cfg`` on all subsequent ``devpi use ...`` operations.

Installing, uploading, testing and releasing
+++++++++++++++++++++++++++++++++++++++++++++++++

You may now continue with install, test and release activities
as described in the :ref:`release process quickstart <quickstart_release_steps>`.

Stopping the server
++++++++++++++++++++++++++++++++++++++++++

Let's not forget to stop our background tutorial server by shutting down supervisor::

    $ supervisorctl -c gen-config/supervisord.conf shutdown
    Shut down
