Skip to content
Snippets Groups Projects
New issue look: Off

Add Debian package to a PPA

    • Open Issue created by username-removed-26331 @grote

    The Debian package should be pushed to a PPA for easy updates for as long as repomaker is not included in Debian itself.

    Build instructions for the package can be found in debian/README.md.

    Designs

    Child items 0

    • No child items are currently open.

    Activity

    • username-removed-26331 mentioned in issue #174 (closed)

      mentioned in issue #174 (closed)

    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      I've started packaging the depends, you can see what I have here: https://launchpad.net/~fdroid/+archive/ubuntu/repomaker/+packages

      Then I'll upload the repomaker package there too.

      Looks like python3-pyqt5.qtwebengine and python3-cryptography (>= 1.4.0) might be hard to backport to Ubunut/xenial LTS. Any flexibility there?

    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      Donno if you know anything about Selenium and phantomjs, the django-js-reverse tests are failing around those:

      $ sudo apt install git-buildpackage
$ gbp clone https://anonscm.debian.org/git/python-modules/packages/django-js-reverse.git
$ cd django-js-reverse
$ gbp buildpackage -uc -us
[snip]
I: pybuild base:184: cd /build/django-js-reverse-0.7.3/.pybuild/pythonX.Y_2.7/build; python2.7 -m unittest discover -v 
tests.test_urls (unittest.loader.ModuleImportFailure) ... ERROR

======================================================================
ERROR: tests.test_urls (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests.test_urls
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/loader.py", line 254, in _find_tests
    module = self._get_module_from_name(name)
  File "/usr/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
    __import__(name)
  File "tests/__init__.py", line 2, in <module>
    from tests.unit_tests import JSReverseViewTestCaseMinified, JSReverseViewTestCaseNotMinified, \
  File "tests/unit_tests.py", line 30, in <module>
    class AbstractJSReverseTestCase(object):
  File "tests/unit_tests.py", line 32, in AbstractJSReverseTestCase
    selenium = WebDriver()
  File "/usr/lib/python2.7/dist-packages/selenium/webdriver/phantomjs/webdriver.py", line 51, in __init__
    self.service.start()
  File "/usr/lib/python2.7/dist-packages/selenium/webdriver/common/service.py", line 88, in start
    raise WebDriverException("Can not connect to the Service %s" % self.path)
WebDriverException: Message: Can not connect to the Service phantomjs
    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      And the last one for the evening, do you know which lib provides this:

      
I: pybuild base:184: cd /build/django-hvad-1.8.0/.pybuild/pythonX.Y_3.6/build; python3.6 -m unittest discover -v 
hvad.contrib.restframework (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: hvad.contrib.restframework (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: hvad.contrib.restframework
Traceback (most recent call last):
  File "/usr/lib/python3.6/unittest/loader.py", line 462, in _find_test_path
    package = self._get_module_from_name(name)
  File "/usr/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/build/django-hvad-1.8.0/.pybuild/pythonX.Y_3.6/build/hvad/contrib/restframework/__init__.py", line 1, in <module>
    from hvad.contrib.restframework.serializers import (
  File "/build/django-hvad-1.8.0/.pybuild/pythonX.Y_3.6/build/hvad/contrib/restframework/serializers.py", line 3, in <module>
    from rest_framework import serializers
ModuleNotFoundError: No module named 'rest_framework'
    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      Looks like django-hvad's tests require Django Rest Framework which does not seem to be packaged nor in Debian.

    • username-removed-26331
      username-removed-26331 @grote ·
      Author Maintainer

      I've started packaging the depends, you can see what I have here: https://launchpad.net/~fdroid/+archive/ubuntu/repomaker/+packages

      Great news!

      Then I'll upload the repomaker package there too.

      Is #183 a requirement for that?

      Looks like python3-pyqt5.qtwebengine and python3-cryptography (>= 1.4.0) might be hard to backport to Ubunut/xenial LTS. Any flexibility there?

      python3-pyqt5.qtwebengine is needed to show repomaker like a local application with a window. You could try to use the PyQt4 equivalent of it and see if repomaker falls back gracefully to Qt4.

      python3-cryptography (>= 1.4.0) is required for creating SSH keys if I remember correctly. We could create those differently, but I am not sure it is worth the effort.

      WebDriverException: Message: Can not connect to the Service phantomjs

      Looks like the tests expect a phantomjs service to be running. Maybe it is easy to set one up?

      Looks like django-hvad's tests require Django Rest Framework which does not seem to be packaged nor in Debian.

      Arghh... I didn't notice they vendorize code when choosing this library.

      However, it looks like it is at least packaged for pypi. Did you try using that instead of their vendorized version?

    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      • What's the Qt4 equivalent? python3-pyqt4.qtwebengine does not exist.

      • its easy to create SSH keys, just call ssh-keygen.

      • if setup.py doesn't set up the tests, then I'll just patch out those tests. Unless you want to write a patch to make setup.py do it.

      • FYI, python3-tinymce also includes a vendorized/minimized copy of tinymce. The Debian package of that must use the tinymce Debian package, but that's at v3.4.8.

    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      Here's another test setup failure, if you have any ideas:

      I: pybuild base:184: cd /build/django-background-tasks-1.1.11/.pybuild/pythonX.Y_2.7/build; python2.7 -m unittest discover -v 
background_task.tests.test_tasks (unittest.loader.ModuleImportFailure) ... ERROR

======================================================================
ERROR: background_task.tests.test_tasks (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: background_task.tests.test_tasks
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/loader.py", line 254, in _find_tests
    module = self._get_module_from_name(name)
  File "/usr/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
    __import__(name)
  File "background_task/tests/test_tasks.py", line 6, in <module>
    from django.contrib.auth.models import User
  File "/usr/lib/python2.7/dist-packages/django/contrib/auth/models.py", line 4, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/usr/lib/python2.7/dist-packages/django/contrib/auth/base_user.py", line 52, in <module>
    class AbstractBaseUser(models.Model):
  File "/usr/lib/python2.7/dist-packages/django/contrib/auth/base_user.py", line 53, in AbstractBaseUser
    password = models.CharField(_('password'), max_length=128)
  File "/usr/lib/python2.7/dist-packages/django/db/models/fields/__init__.py", line 1061, in __init__
    super(CharField, self).__init__(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/django/db/models/fields/__init__.py", line 172, in __init__
    self.db_tablespace = db_tablespace or settings.DEFAULT_INDEX_TABLESPACE
  File "/usr/lib/python2.7/dist-packages/django/conf/__init__.py", line 56, in __getattr__
    self._setup(name)
  File "/usr/lib/python2.7/dist-packages/django/conf/__init__.py", line 39, in _setup
    % (desc, ENVIRONMENT_VARIABLE))
ImproperlyConfigured: Requested setting DEFAULT_INDEX_TABLESPACE, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
    • username-removed-26331
      username-removed-26331 @grote ·
      Author Maintainer

      What's the Qt4 equivalent? python3-pyqt4.qtwebengine does not exist.

      You could try to just use python3-pyqt4. It seems to depend on libqtwebkit4 at least.

      its easy to create SSH keys, just call ssh-keygen.

      I know it is easy, but if a library that we include anyway provides the same functionality I would always prefer this to calling yet another external process. Less attack surface, no need to check if the external program exists and proper exceptions instead of parsing process results.

      Can't you update python-cryptography in trusty?

      if setup.py doesn't set up the tests, then I'll just patch out those tests. Unless you want to write a patch to make setup.py do it.

      Do you mean repomaker's setup.py? Unfortunately, the django test runner doesn't play well with setuptools. I had briefly looked into it. There are some third-party libraries that can make the connection, but they are rather hacky, so had decided against them for now.

      FYI, python3-tinymce also includes a vendorized/minimized copy of tinymce. The Debian package of that must use the tinymce Debian package, but that's at v3.4.8.

      We are not using this vendorized copy in repomaker, but install the latest version with npm (like the other web dependencies).

      ImproperlyConfigured: Requested setting DEFAULT_INDEX_TABLESPACE, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

      Looks like the same problem that repomaker is having. You are trying to run django tests with setuptools which doesn't work without the proper django initialization magic.

    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      So this is Ubuntu/xenial 16.04 we're talking about. Ubuntu Trusty 14.04 would be a lot more work.

      Backporting packages is always possible, e.g. python-cryptography, but each backported package makes it more likely that the PPA will conflict with other uses, since it'll upgrade those packages.

      For now, I'll just disable the tests when they fail.

      As for trying python3-pyqt4, I wouldn't know how.

    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      Ok, I think I have all Repomaker dependencies in the PPA for Ubuntu/xenial and newer. Please try it and let me know: https://launchpad.net/%7Efdroid/+archive/ubuntu/repomaker/+packages

      # add-apt-repository ppa:fdroid-repomaker
# echo signing key 9AAC 2531 93B6 5D4D F1D0 A13E EC46 32C7 9C5E 0151
# apt update
# apt install ...

      I don't know how to build that dh-virtualenv package. Maybe that's no longer needed?

    • username-removed-26331
      username-removed-26331 @grote ·
      Author Maintainer

      You mean

      # add-apt-repository ppa:fdroid/repomaker

      Repomaker itself doesn't seem to be part of that repo, right?

      I don't know how to build that dh-virtualenv package. Maybe that's no longer needed?

      No, this is just to build the debian package with a python virtualenv to provide the missing dependencies (without the ppa).

    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      ok, I stripped out the virtualenv stuff and uploaded the source package to the PPA. We'll see if it builds...

    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      it failed here:

         dh_auto_build -O--buildsystem=pybuild
I: pybuild base:184: /usr/bin/python3 setup.py build 
Packages: ['repomaker', 'repomaker.migrations', 'repomaker.models', 'repomaker.views']
/usr/lib/python3/dist-packages/setuptools/dist.py:341: UserWarning: Normalizing '0.0.2a' to '0.0.2a0'
  normalized_version,
running build
running build_py
creating /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/settings_test.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/settings.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/settings_desktop.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/storage.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/gui.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/context_processors.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/__init__.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/wsgi.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/settings_test_multi_user.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/urls.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/utils.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/tasks.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/admin.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
copying repomaker/apps.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker
creating /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/migrations
copying repomaker/migrations/default_remote_repositories.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/migrations
copying repomaker/migrations/0001_initial.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/migrations
copying repomaker/migrations/default_categories.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/migrations
copying repomaker/migrations/__init__.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/migrations
copying repomaker/migrations/default_user.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/migrations
creating /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/models
copying repomaker/models/remoterepository.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/models
copying repomaker/models/app.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/models
copying repomaker/models/remoteapp.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/models
copying repomaker/models/category.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/models
copying repomaker/models/storage.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/models
copying repomaker/models/repository.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/models
copying repomaker/models/apkpointer.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/models
copying repomaker/models/__init__.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/models
copying repomaker/models/screenshot.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/models
copying repomaker/models/apk.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/models
creating /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/views
copying repomaker/views/remoterepository.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/views
copying repomaker/views/gitstorage.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/views
copying repomaker/views/app.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/views
copying repomaker/views/storage.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/views
copying repomaker/views/repository.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/views
copying repomaker/views/__init__.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/views
copying repomaker/views/screenshot.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/views
copying repomaker/views/apk.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/views
copying repomaker/views/sshstorage.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/views
copying repomaker/views/s3storage.py -> /build/repomaker-0.0.4/.pybuild/pythonX.Y_3.5/build/repomaker/views
error: package directory 'repomaker-static' does not exist
E: pybuild pybuild:283: build: plugin distutils failed with: exit code=1: /usr/bin/python3 setup.py build 
dh_auto_build: pybuild --build -i python{version} -p 3.5 returned exit code 13
debian/rules:7: recipe for target 'build' failed
make: *** [build] Error 25

      Here's the patch I used to build the Debian package: 0001-flush-out-Debian-packaging-in-proper-DPMT-style.patch

    • username-removed-26331
      username-removed-26331 @grote ·
      Author Maintainer

      error: package directory 'repomaker-static' does not exist

      Did you run the pre-build script to fetch the static web resources? It is mentioned in the instructions: https://gitlab.com/fdroid/repomaker/blob/master/debian/README.md#building-debian-package

    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      no, I just built the package with dpkg-buildpackage. In order to have it on the PPA, it'll have to build with all the sources bundled somehow. The Ubuntu/Debian builders do not allow network access during the build process. If this is a lot of work, we can stick to your .deb builds for now, but that won't be in a PPA.

    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      To get it into our PPA, we would just need to bundle the node libs in the source tarball for Repomaker.

      We'll have to ultimately package the node stuff for Debian to get this into Debian. Given that there were tons and tons of node packages added to Debian/stretch, I imagine its gotten quite easy to do.

    • username-removed-26331
      username-removed-26331 @grote ·
      Author Maintainer

      As for trying python3-pyqt4, I wouldn't know how.

      Just depend on this instead of the other Qt5 dependencies for new versions.

      To get it into our PPA, we would just need to bundle the node libs in the source tarball for Repomaker.

      Yes, this is what the pre-release script is doing among other things. Essentially it creates this repomaker-static package as expected by the installed application.

      The Ubuntu/Debian builders do not allow network access during the build process.

      If I understood you correctly, the ppa builders do not have this restriction, right?

      When uploading repomaker to pypi, I was planning on including the repomaker-static module in the upload, so it is ready to be used without needing npm, etc.

      We'll have to ultimately package the node stuff for Debian to get this into Debian. Given that there were tons and tons of node packages added to Debian/stretch, I imagine its gotten quite easy to do.

      Hopefully! I leave this to you then ;)

    • username-removed-24982
      username-removed-24982 @eighthave ·
      Owner

      Just to jot this down from our conversation, it looks like these can be provided by existing packages:

      • roboto-fontface --> fonts-roboto
      • material-design-icons-iconfont --> fonts-materialdesignicons-webfont
      • tinymce --> tinymce

      Leaving these to be embedded as source:

      • dialog-polyfill
      • material-design-lite
    • username-removed-26331 changed milestone to %1.0

      changed milestone to %1.0

    Please register or sign in to reply