Pulp

My commentary about the redis settings not working: if I use master instead of rc1 the env vars work. As do the settings.py vars.

REDIS vars seem working in
commit 1ec10543ee2dcdb4c317a73c94005412f9d9dd04 (HEAD -> master)

Installed via:
git clone https://github.com/pulp/pulpcore.git
pip install -e ./pulpcore[postgres]
git clone https://github.com/pulp/pulpcore-plugin.git
pip install -e ./pulpcore-plugin

I blame: https://docs.pulpproject.org/en/3.0/nightly/installation/instructions.html

(.venv) ubuntu@buildbox:~$ rq worker -n -c pulpcore.rqconfig 'resource-manager@%h' -w 'pulpcore.tasking.worker.PulpWorker'
2019-04-24:13:26:24,811 ERROR [redis_loader.py:58 - load] Error 111 connecting to localhost:6379. Connection refused.
Error 111 connecting to localhost:6379. Connection refused.
(.venv) ubuntu@buildbox:~$ grep REDIS /etc/pulp/settings.py
REDIS_ENABLED_FOR_DYNACONF=True
REDIS_HOST='jps-pulp-test.k5xwqi.ng.0001.use1.cache.amazonaws.com'
REDIS_PORT= 6379

More context. I was stepping thru pulpcore/pulpcore/rqconfig.py and notice it was coughing an error about redis:

This is with DJANGO_SETTINGS_MODULE set.

In [1]: import sys

In [2]: from pulpcore.app.settings import settings
2019-04-24:14:05:30,274 ERROR    [redis_loader.py:58 - load] Error 111 connecting to localhost:6379. Connection refused.

Chasing the bug deeper, I ended up in redis_loader.py line 58 in dynaconf v2.0.1

The line in question is in an except Exception block that just swallows and logs everything. Which explains the ERROR observed above.

Further debugging using the powerful print() function showed I was hitting the else block of the try/catch.

Removing the try/catch block

def load(obj, env=None, silent=True, key=None):
    """Reads and loads in to "settings" a single key or all keys from redis

    :param obj: the settings instance
    :param env: settings env default='DYNACONF'
    :param silent: if errors should raise
    :param key: if defined load a single key, else load all in env
    :return: None
    """
    redis = StrictRedis(**obj.get("REDIS_FOR_DYNACONF"))
    holder = obj.get("ENVVAR_PREFIX_FOR_DYNACONF")
    try:
        if key:
            value = redis.hget(holder.upper(), key)
            if value:
                obj.logger.debug(
                    "redis_loader: loading by key: %s:%s (%s:%s)",
                    key,
                    value,
                    IDENTIFIER,
                    holder,
                )
            if value:
                parsed_value = parse_conf_data(value, tomlfy=True)
                if parsed_value:
                    obj.set(key, parsed_value)
        else:
            data = {
                key: parse_conf_data(value, tomlfy=True)
                for key, value in redis.hgetall(holder.upper()).items()
            }
            if data:
                obj.logger.debug(
                    "redis_loader: loading: %s (%s:%s)",
                    data,
                    IDENTIFIER,
                    holder,
                )
                obj.update(data, loader_identifier=IDENTIFIER)
    except Exception as e:
        if silent:
            if hasattr(obj, "logger"):
                obj.logger.error(str(e))
            return False
        raise

I removed the try/catch and realized i was mis-reading a list comprehension and the root error is it's still trying to connect to localhost.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "x.py", line 1, in <module>
    from pulpcore.app.settings import settings
  File "/home/ubuntu/pulpcore/pulpcore/app/settings.py", line 218, in <module>
    ENVVAR_FOR_DYNACONF='PULP_SETTINGS',
  File "/home/ubuntu/.venv/lib/python3.6/site-packages/dynaconf/contrib/django_dynaconf_v2.py", line 67, in load
    lazy_settings.populate_obj(django_settings_module)
  File "/home/ubuntu/.venv/lib/python3.6/site-packages/dynaconf/base.py", line 90, in __getattr__
    self._setup()
  File "/home/ubuntu/.venv/lib/python3.6/site-packages/dynaconf/base.py", line 121, in _setup
    settings_module=settings_module, **self._kwargs
  File "/home/ubuntu/.venv/lib/python3.6/site-packages/dynaconf/base.py", line 184, in __init__
    self.execute_loaders()
  File "/home/ubuntu/.venv/lib/python3.6/site-packages/dynaconf/base.py", line 736, in execute_loaders
    loader.load(self, env, silent=silent, key=key)
  File "/home/ubuntu/.venv/lib/python3.6/site-packages/dynaconf/loaders/redis_loader.py", line 44, in load
    for key, value in redis.hgetall(holder.upper()).items()
  File "/home/ubuntu/.venv/lib/python3.6/site-packages/redis/client.py", line 2649, in hgetall
    return self.execute_command('HGETALL', name)
  File "/home/ubuntu/.venv/lib/python3.6/site-packages/redis/client.py", line 774, in execute_command
    connection.send_command(*args)
  File "/home/ubuntu/.venv/lib/python3.6/site-packages/redis/connection.py", line 619, in send_command
    self.send_packed_command(self.pack_command(*args))
  File "/home/ubuntu/.venv/lib/python3.6/site-packages/redis/connection.py", line 594, in send_packed_command
    self.connect()
  File "/home/ubuntu/.venv/lib/python3.6/site-packages/redis/connection.py", line 498, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to localhost:6379. Connection refused.

update for pulplift is here: https://github.com/pulp/pulplift/pull/28