Well, they're not yet compiled, but in the next commit
they should be.
People have been complaining [citation needed] to me
via email or via Github that Python's performance is crap
because it packages sources directly and they're not compiled.
And Python has to compile the sources on each run, and
on-the-fly.
Allowing compilation caching is also a no-no, because
I'll get complaints that the flash storage fills up
whenever a Python app runs.
So, to give the user a choice, the new de-facto packaging
for Python packages will be:
* ship compiled + [ preferably ] optimized files
* package sources separately
The problem is that this doubles the number of packages
in LEDE/OpenWrt, but build-times should not suffer a big
hit, since the compilation is done once, and the
install phase should not be too intensive.
Oh, and people don't need ship source packages if
they don't want to.
To do that, a packager needs to just call
`$(eval $(call BuildPackage,python-<package>-src))`
The `python-` prefix is important.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
I just found out that, that the BUILD_VARIANT var
is not set for the host build, so technically this code
would never get used.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
As both LEDE and OpenWrt have STAGING_DIR_HOSTPKG now, we can start to rely
on it. See 73b7f55424 for more information on
STAGING_DIR_HOSTPKG.
STAGING_DIR_HOSTPKG won't actually be changed before the first LEDE release
(it is equivalent to $(STAGING_DIR)/host), so this simple search/replace
cleanup is safe to apply. Doing this cleanup now will be useful for the
Gluon project (an OpenWrt/LEDE based firmware framework) for experimenting
with modifying STAGING_DIR_HOSTPKG before doing this in the LEDE upstream.
Also fixes a typo in the dbus Makefile ("STAGIND_DIR").
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
STAGING_DIR_HOSTPKG is now defined in both OpenWrt and LEDE, so we can
start to rely on it.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Most python packages use the same build rules & vars.
So, adding them in python-package.mk.
Also, preparing for using VARIANTs for python/python3 packages.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
--sysconfigdir and --sbindir are obtained from --prefix
and --exec-prefix, so no need to have them explicitly
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Allows for more granularity with respect to python-host.mk
and python-package.mk inclusions.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
So that it can be included twice.
python-package.mk include python-host.mk, but at some point
in time python-host.mk is not exported to staging_dir yet.
So, to have in python's Makefile, we need to include it
in the Makefile, and to prevent double inclusion (if
python-host.mk is exported from a previous build), the
include guard is added.
Also, moved the includes after some vars were set.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Currently, the python-host.mk framework relies on HOST_BUILD_PREFIX to
refer to the $(STAGING_DIR)/host directory but using the HOST_BUILD_PREFIX
variable requires the use of include/host-build.mk which in turn includes
python-host.mk, leading to target redefinition errors.
In order to provide a global, uniform way to refer to the host staging
directory, LEDE introduced a new variable STAGING_DIR_HOSTPKG which points
to $(STAGING_DIR)/host for now with the purpose of eventually being able to
relocate that directory in the future.
This commit changes python-host.mk to ...
- stop including include/host-build.mk (revert of #3423)
- replace usages of $(HOST_BUILD_PREFIX) with $(STAGING_DIR_HOSTPKG)
- warn and fallback to $(STAGING_DIR)/host if STAGING_DIR_HOSTPKG is
unavailable
The fallback code will ensure that the python host build infrastructure
continues to work properly on older OpenWrt and LEDE versions until the
STAGING_DIR_HOSTPKG is fully settled in and can be removed some time
in the future.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Also, override all prefix args in the HOST_CONFIGURE_ARGS
so that this works fine on CC/15.05.
There are some changes in core regarding package builds that
require this.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
It seems that this can become an issue.
Found when building python3-setuptools from packages/trunk
in openwrt/15.05.
python3-setuptools is not in packages/for-15.05
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
The sets host Python's prefix and exec-prefix to the same value
($(STAGING_DIR)/host for packages, as currently defined in
include/host-build.mk), which avoids the case where pyconfig.h is
not in the same location as Python's other header files (see
https://bugs.python.org/issue896330).
This also removes some cross compilation workarounds
(_python_sysroot/prefix/exec_prefix, disabling byte-compilation,
__PYVENV_LAUNCHER__) that are not necessary when compiling packages
for host.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
AFAIK sysroot is combined with prefix/exec_prefix, so sysroot shouldn't
include "/usr". (cf. the HostPython function in older versions of
python-package.mk)
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This adds code to make it easier to build third-party python
modules such as ones from PyPi.
Signed-off-by: Daniel Dickinson <openwrt@daniel.thecshore.com>
And split python-dev and python-lib2to3 packages.
Seems for python3 this was done, so we'll sync here.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
That means basically moving the host Python from $(STAGING_DIR_HOST)
to $(STAGING_DIR_HOST)/usr and making the PYTHONPATH var
as an argument.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Some python packages (e.g. cffi) compile one or more shared libraries
as part of their setup process. When these packages are setup
dependencies of other packages (e.g. cryptography), these packages (and
their shared libraries) will need to be loaded on the host system.
This adds a makefile, similar to python-package.mk, to simplify
installing python packages on the host.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
python-package.mk now defines (extends) CONFIGURE_ARGS that are
adjusting PREFIX and EXEC_PREFIX in distutils.sysconfig during
compilation. These variables are sometimes used by autotools to detect
path to Python header files. Adding these variables to python-package.mk
fixes compilation of legacy autotools-configured packages that already
include python-package.mk and do not overwrite CONFIGURE_ARGS.
Signed-off-by: Jan Čermák <jan.cermak@nic.cz>
Cc: Alexandru Ardelean <ardeleanalex@gmail.com>
Rule of thumb is: any Python file that is greater than 100kb
(or adds a dependency with which it adds more than 100 kb)
should be a pretty useful/commonly used lib to stay in `python-light`.
An example, is the Python IO lib, which summarized (Python source +
binary module) is over 200kb.
Also moved some files that should have been put into previously
existing packages before, and re-organized the packages a bit.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Packetize some Python modules to reduce size of the overall Python package.
Basically, a new package is introduced, call `python-light`,
and everything that's extra (or big) is put in other packages.
The `python` package becomes a metapackage that installs `python-light`
along with the rest of the packages.
Base work started by Jan Čermák.
Signed-off-by: Jan Čermák <jan.cermak@nic.cz>
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
That would allow for files (in filespecs) to be indented.
As it is now, the files need to be added at the begginning of the line.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Alexandru Ardelean
Matthias Schiffer
Jo-Philipp Wich
Yousong Zhou
Jeffery To
Felix Fietkau
Daniel Dickinson
Attila Lendvai
Jan Čermák
Alexandru Ardelean