Nectar

Randy are you sure it is not your env problem?
My tests all pass on master:

$ ./run-tests.sh -x
...........Exception AttributeError: "'DownloaderConfig' object has no attribute '_temp_files'" in <bound method DownloaderConfig.__del__ of <nectar.config.DownloaderConfig object at 0x7f8cc64dd790>> ignored
.............................127.0.0.1 - - [03/Dec/2015 12:22:20] "GET /test/unit/data/100K_file HTTP/1.1" 200 -
127.0.0.1 - - [03/Dec/2015 12:22:20] "GET /test/unit/data/500K_file HTTP/1.1" 200 -
127.0.0.1 - - [03/Dec/2015 12:22:20] "GET /test/unit/data/1M_file HTTP/1.1" 200 -
127.0.0.1 - - [03/Dec/2015 12:22:20] code 404, message File not found
127.0.0.1 - - [03/Dec/2015 12:22:20] "GET /test/unit/data/notme HTTP/1.1" 404 -
127.0.0.1 - - [03/Dec/2015 12:22:20] code 404, message File not found
127.0.0.1 - - [03/Dec/2015 12:22:20] "GET /test/unit/data/notmeeither HTTP/1.1" 404 -
.127.0.0.1 - - [03/Dec/2015 12:22:21] code 404, message File not found
127.0.0.1 - - [03/Dec/2015 12:22:21] "GET /test/unit/data/idontexistanddontcreateme HTTP/1.1" 404 -
.127.0.0.1 - - [03/Dec/2015 12:22:21] "GET /test/unit/data/100K_file HTTP/1.1" 200 -
.127.0.0.1 - - [03/Dec/2015 12:22:22] "GET /test/unit/data/500K_file HTTP/1.1" 200 -
.........
Name                          Stmts   Miss  Cover   Missing
-----------------------------------------------------------
nectar                            0      0   100%   
nectar.config                    74      0   100%   
nectar.downloaders                0      0   100%   
nectar.downloaders.base          36      3    92%   50, 146-147
nectar.downloaders.local        109     10    91%   38-41, 161-162, 168-171
nectar.downloaders.threaded     253     10    96%   145, 209, 265-266, 323-326, 404, 437
nectar.listener                  22      3    86%   31, 34, 59
nectar.report                    53      3    94%   86, 110, 141
nectar.request                   17      0   100%   
-----------------------------------------------------------
TOTAL                           564     29    95%   
----------------------------------------------------------------------
Ran 52 tests in 6.621s

OK

ipanova@redhat.com wrote:

Randy are you sure it is not your env problem?

This is on a brand new Fedora 23 Vagrant environment:

[rbarlow@how pulp]$ vagrant ssh
Last login: Thu Dec  3 19:32:40 2015 from 192.168.121.1

Welcome to the Pulp dev environment!

Here are some tips:
* Your code is all checked out inside of ~/devel/.

* The default username:password is admin:admin. When your session expires, you
  can log in again with pulp-admin login -u admin

* You can type workon <project> to quickly cd to a project dir and activate its
  virtualenv. For example:

    $ workon pulp_python

  Similarly, run "workon pulp" to jump into the pulp platform virtualenv.

* Each project has a "run-tests.py" to run the unit tests.

* There are a set of bash functions in your .bashrc that are useful:

    pstart:    Start all Pulp services
    pstop:     Stop all Pulp services
    prestart:  Restart all Pulp services
    pstatus:   Report status on all Pulp services
    ptests:    Run all unit test on all the Pulp repositories
    psmash:    Run Pulp Smash!
    preset:    Reset Pulp to a clean install state
    ppopulate: Populate Pulp with test repositories of several types

* You can ssh into your vagrant environment with vagrant ssh, but presumably
  you already know this since you are reading this message :)

More information can be found at
https://pulp.readthedocs.org/en/latest/dev-guide/ and you can ask questions in
#pulp on Freenode IRC.

Happy hacking, and thanks for your contribution!

[vagrant@dev ~]$ cd devel/nectar/
[vagrant@dev nectar]$ mkvirtualenv --system-site-packages nectar
New python executable in nectar/bin/python2
Also creating executable in nectar/bin/python
Installing setuptools, pip...done.
(nectar)[vagrant@dev nectar]$ pip install nose mock coverage
You are using pip version 6.0.8, however version 7.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting nose
  Using cached nose-1.3.7-py2-none-any.whl
Collecting mock
  Using cached mock-1.3.0-py2.py3-none-any.whl
Collecting coverage
  Using cached coverage-4.0.3.tar.gz
Collecting funcsigs (from mock)
  Using cached funcsigs-0.4-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): pbr>=0.11 in /usr/lib/python2.7/site-packages (from mock)
Requirement already satisfied (use --upgrade to upgrade): six>=1.7 in /usr/lib/python2.7/site-packages (from mock)
Installing collected packages: funcsigs, coverage, mock, nose

  Running setup.py install for coverage
    building 'coverage.tracer' extension
    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c coverage/ctracer/datastack.c -o build/temp.linux-x86_64-2.7/coverage/ctracer/datastack.o
    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c coverage/ctracer/filedisp.c -o build/temp.linux-x86_64-2.7/coverage/ctracer/filedisp.o
    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c coverage/ctracer/module.c -o build/temp.linux-x86_64-2.7/coverage/ctracer/module.o
    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c coverage/ctracer/tracer.c -o build/temp.linux-x86_64-2.7/coverage/ctracer/tracer.o
    gcc -pthread -shared -Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld build/temp.linux-x86_64-2.7/coverage/ctracer/datastack.o build/temp.linux-x86_64-2.7/coverage/ctracer/filedisp.o build/temp.linux-x86_64-2.7/coverage/ctracer/module.o build/temp.linux-x86_64-2.7/coverage/ctracer/tracer.o -L/usr/lib64 -lpython2.7 -o build/lib.linux-x86_64-2.7/coverage/tracer.so
    Installing coverage2 script to /home/vagrant/.virtualenvs/nectar/bin
    Installing coverage-2.7 script to /home/vagrant/.virtualenvs/nectar/bin
    Installing coverage script to /home/vagrant/.virtualenvs/nectar/bin

Successfully installed coverage-4.0.3 funcsigs-0.4 mock-1.3.0 nose-1.3.7
(nectar)[vagrant@dev nectar]$ ./run-tests.sh -x
...........Exception AttributeError: "'DownloaderConfig' object has no attribute '_temp_files'" in <bound method DownloaderConfig.__del__ of <nectar.config.DownloaderConfig object at 0x7fa98d7fccd0>> ignored
.............................F.FF........
======================================================================
FAIL: test_multiple_downloads (test_threaded_downloader.LiveDownloadingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vagrant/devel/nectar/test/unit/test_threaded_downloader.py", line 188, in test_multiple_downloads
    self.assertEqual(len(lst.succeeded_reports), len(self.data_file_names))
AssertionError: 0 != 3
-------------------- >> begin captured logging << --------------------
nectar.downloaders.threaded: DEBUG: starting workers
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
nectar.downloaders.threaded: WARNING: Connection Error - http://localhost:8088/test/unit/data/100K_file could not be reached.
nectar.downloaders.threaded: DEBUG: Skipping http://localhost:8088/test/unit/data/500K_file because localhost:8088 could not be reached.
nectar.downloaders.threaded: DEBUG: Skipping http://localhost:8088/test/unit/data/1M_file because localhost:8088 could not be reached.
nectar.downloaders.threaded: DEBUG: Skipping http://localhost:8088/test/unit/data/notme because localhost:8088 could not be reached.
nectar.downloaders.threaded: DEBUG: Skipping http://localhost:8088/test/unit/data/notmeeither because localhost:8088 could not be reached.
--------------------- >> end captured logging << ---------------------

======================================================================
FAIL: test_single_download_success (test_threaded_downloader.LiveDownloadingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vagrant/devel/nectar/test/unit/test_threaded_downloader.py", line 130, in test_single_download_success
    self.assertTrue(os.path.exists(dest_path))
AssertionError: False is not true
-------------------- >> begin captured logging << --------------------
nectar.downloaders.threaded: DEBUG: starting workers
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
nectar.downloaders.threaded: WARNING: Connection Error - http://localhost:8088/test/unit/data/100K_file could not be reached.
--------------------- >> end captured logging << ---------------------

======================================================================
FAIL: test_throttling (test_threaded_downloader.LiveDownloadingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vagrant/devel/nectar/test/unit/test_threaded_downloader.py", line 217, in test_throttling
    self.assertTrue(finish - start >= two_seconds)
AssertionError: False is not true
-------------------- >> begin captured logging << --------------------
nectar.downloaders.threaded: DEBUG: starting workers
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
nectar.downloaders.threaded: WARNING: Connection Error - http://localhost:8088/test/unit/data/500K_file could not be reached.
--------------------- >> end captured logging << ---------------------

Name                             Stmts   Miss  Cover   Missing
--------------------------------------------------------------
nectar.py                            0      0   100%   
nectar/config.py                    74      0   100%   
nectar/downloaders.py                0      0   100%   
nectar/downloaders/base.py          36      3    92%   50, 146-147
nectar/downloaders/local.py        109     10    91%   38-41, 161-162, 168-171
nectar/downloaders/threaded.py     253     13    95%   145, 209, 265-266, 270-271, 279, 323-326, 404, 437
nectar/listener.py                  22      3    86%   31, 34, 59
nectar/report.py                    53      3    94%   86, 110, 141
nectar/request.py                   17      0   100%   
--------------------------------------------------------------
TOTAL                              564     32    94%   
----------------------------------------------------------------------
Ran 52 tests in 5.261s

FAILED (failures=3)

Randy, I don't know yet where exactly stands the issue, but i fount out what service provides connection to localhost with port 8088.
https://github.com/pulp/nectar/blob/master/test/unit/test_threaded_downloader.py#L101
https://github.com/pulp/nectar/blob/master/test/utils/http_static_test_server.py#L14

Before running tests:

$ telnet localhost 8088
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused

During time tests are running,when this static server is enabled:

$ telnet localhost 8088
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

Probably it worth to check ports and firewall

With the help of Ina (thanks!) we've found the problem. The unit tests start a server that listens only on 127.0.0.1, but the name "localhost" on F23 only refers to ::1 now. Basically, the test server does IPv4, and the unit tests try to connect to it with IPv6 only.

To resolve this, we just need to figure out how to get the server to listen on IPv6.