python: split source packages away from compiled packages
Well, they're not yet compiled, but in the next commit
they should be.
People have been complaining [citation needed] to me
via email or via Github that Python's performance is crap
because it packages sources directly and they're not compiled.
And Python has to compile the sources on each run, and
on-the-fly.
Allowing compilation caching is also a no-no, because
I'll get complaints that the flash storage fills up
whenever a Python app runs.
So, to give the user a choice, the new de-facto packaging
for Python packages will be:
* ship compiled + [ preferably ] optimized files
* package sources separately
The problem is that this doubles the number of packages
in LEDE/OpenWrt, but build-times should not suffer a big
hit, since the compilation is done once, and the
install phase should not be too intensive.
Oh, and people don't need ship source packages if
they don't want to.
To do that, a packager needs to just call
`$(eval $(call BuildPackage,python-<package>-src))`
The `python-` prefix is important.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
8 years ago python: split source packages away from compiled packages
Well, they're not yet compiled, but in the next commit
they should be.
People have been complaining [citation needed] to me
via email or via Github that Python's performance is crap
because it packages sources directly and they're not compiled.
And Python has to compile the sources on each run, and
on-the-fly.
Allowing compilation caching is also a no-no, because
I'll get complaints that the flash storage fills up
whenever a Python app runs.
So, to give the user a choice, the new de-facto packaging
for Python packages will be:
* ship compiled + [ preferably ] optimized files
* package sources separately
The problem is that this doubles the number of packages
in LEDE/OpenWrt, but build-times should not suffer a big
hit, since the compilation is done once, and the
install phase should not be too intensive.
Oh, and people don't need ship source packages if
they don't want to.
To do that, a packager needs to just call
`$(eval $(call BuildPackage,python-<package>-src))`
The `python-` prefix is important.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
8 years ago python: split source packages away from compiled packages
Well, they're not yet compiled, but in the next commit
they should be.
People have been complaining [citation needed] to me
via email or via Github that Python's performance is crap
because it packages sources directly and they're not compiled.
And Python has to compile the sources on each run, and
on-the-fly.
Allowing compilation caching is also a no-no, because
I'll get complaints that the flash storage fills up
whenever a Python app runs.
So, to give the user a choice, the new de-facto packaging
for Python packages will be:
* ship compiled + [ preferably ] optimized files
* package sources separately
The problem is that this doubles the number of packages
in LEDE/OpenWrt, but build-times should not suffer a big
hit, since the compilation is done once, and the
install phase should not be too intensive.
Oh, and people don't need ship source packages if
they don't want to.
To do that, a packager needs to just call
`$(eval $(call BuildPackage,python-<package>-src))`
The `python-` prefix is important.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
8 years ago python: split source packages away from compiled packages
Well, they're not yet compiled, but in the next commit
they should be.
People have been complaining [citation needed] to me
via email or via Github that Python's performance is crap
because it packages sources directly and they're not compiled.
And Python has to compile the sources on each run, and
on-the-fly.
Allowing compilation caching is also a no-no, because
I'll get complaints that the flash storage fills up
whenever a Python app runs.
So, to give the user a choice, the new de-facto packaging
for Python packages will be:
* ship compiled + [ preferably ] optimized files
* package sources separately
The problem is that this doubles the number of packages
in LEDE/OpenWrt, but build-times should not suffer a big
hit, since the compilation is done once, and the
install phase should not be too intensive.
Oh, and people don't need ship source packages if
they don't want to.
To do that, a packager needs to just call
`$(eval $(call BuildPackage,python-<package>-src))`
The `python-` prefix is important.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
8 years ago python: split source packages away from compiled packages
Well, they're not yet compiled, but in the next commit
they should be.
People have been complaining [citation needed] to me
via email or via Github that Python's performance is crap
because it packages sources directly and they're not compiled.
And Python has to compile the sources on each run, and
on-the-fly.
Allowing compilation caching is also a no-no, because
I'll get complaints that the flash storage fills up
whenever a Python app runs.
So, to give the user a choice, the new de-facto packaging
for Python packages will be:
* ship compiled + [ preferably ] optimized files
* package sources separately
The problem is that this doubles the number of packages
in LEDE/OpenWrt, but build-times should not suffer a big
hit, since the compilation is done once, and the
install phase should not be too intensive.
Oh, and people don't need ship source packages if
they don't want to.
To do that, a packager needs to just call
`$(eval $(call BuildPackage,python-<package>-src))`
The `python-` prefix is important.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
8 years ago python: split source packages away from compiled packages
Well, they're not yet compiled, but in the next commit
they should be.
People have been complaining [citation needed] to me
via email or via Github that Python's performance is crap
because it packages sources directly and they're not compiled.
And Python has to compile the sources on each run, and
on-the-fly.
Allowing compilation caching is also a no-no, because
I'll get complaints that the flash storage fills up
whenever a Python app runs.
So, to give the user a choice, the new de-facto packaging
for Python packages will be:
* ship compiled + [ preferably ] optimized files
* package sources separately
The problem is that this doubles the number of packages
in LEDE/OpenWrt, but build-times should not suffer a big
hit, since the compilation is done once, and the
install phase should not be too intensive.
Oh, and people don't need ship source packages if
they don't want to.
To do that, a packager needs to just call
`$(eval $(call BuildPackage,python-<package>-src))`
The `python-` prefix is important.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
8 years ago python: split source packages away from compiled packages
Well, they're not yet compiled, but in the next commit
they should be.
People have been complaining [citation needed] to me
via email or via Github that Python's performance is crap
because it packages sources directly and they're not compiled.
And Python has to compile the sources on each run, and
on-the-fly.
Allowing compilation caching is also a no-no, because
I'll get complaints that the flash storage fills up
whenever a Python app runs.
So, to give the user a choice, the new de-facto packaging
for Python packages will be:
* ship compiled + [ preferably ] optimized files
* package sources separately
The problem is that this doubles the number of packages
in LEDE/OpenWrt, but build-times should not suffer a big
hit, since the compilation is done once, and the
install phase should not be too intensive.
Oh, and people don't need ship source packages if
they don't want to.
To do that, a packager needs to just call
`$(eval $(call BuildPackage,python-<package>-src))`
The `python-` prefix is important.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
8 years ago |
|
- #!/bin/sh
- set -e
-
- [ -z "$SOURCE_DATE_EPOCH" ] || {
- PYTHONHASHSEED="$SOURCE_DATE_EPOCH"
- export PYTHONHASHSEED
- }
-
- process_filespec() {
- local src_dir="$1"
- local dst_dir="$2"
- local filespec="$3"
- echo "$filespec" | (
- IFS='|'
- while read fop fspec fperm; do
- local fop=`echo "$fop" | tr -d ' \t\n'`
- if [ "$fop" = "+" ]; then
- if [ ! -e "${src_dir}${fspec}" ]; then
- echo "File not found '${src_dir}${fspec}'"
- exit 1
- fi
- dpath=`dirname "$fspec"`
- if [ -z "$fperm" ]; then
- dperm=`stat -c "%a" ${src_dir}${dpath}`
- fi
- mkdir -p -m$dperm ${dst_dir}${dpath}
- echo "copying: '$fspec'"
- cp -fpR ${src_dir}${fspec} ${dst_dir}${dpath}/
- if [ -n "$fperm" ]; then
- chmod -R $fperm ${dst_dir}${fspec}
- fi
- elif [ "$fop" = "-" ]; then
- echo "removing: '$fspec'"
- rm -fR ${dst_dir}${fspec}
- elif [ "$fop" = "=" ]; then
- echo "setting permissions: '$fperm' on '$fspec'"
- chmod -R $fperm ${dst_dir}${fspec}
- fi
- done
- )
- }
-
- delete_empty_dirs() {
- local dst_dir="$1"
- if [ -d "$dst_dir/usr" ] ; then
- for _ in $(seq 1 10) ; do
- find "$dst_dir/usr" -empty -type d -exec rmdir {} \; || continue
- break
- done
- rmdir "$dst_dir/usr" || true
- fi
- }
-
- ver="$1"
- src_dir="$2"
- dst_dir="$3"
- python="$4"
- mode="$5"
- filespec="$6"
-
- SED="${SED:-sed -e}"
-
- find "$src_dir" -name "*\.exe" -exec rm -f {} \;
-
- process_filespec "$src_dir" "$dst_dir" "$filespec" || {
- echo "process filespec error-ed"
- exit 1
- }
-
- usr_bin_dir="$dst_dir/usr/bin"
-
- if [ -d "$usr_bin_dir" ] ; then
- $SED "1"'!'"b;s,^#"'!'".*python.*,#"'!'"/usr/bin/python${ver}," -i --follow-symlinks $usr_bin_dir/*
- fi
-
- if [ "$mode" == "sources" ] ; then
- # Copy only python source files
- find "$dst_dir" -not -type d -not -name "*\.py" -exec rm -f {} \;
-
- delete_empty_dirs "$dst_dir"
- exit 0
- fi
-
- legacy=
- [ "$ver" == "3" ] && legacy="-b"
-
- # XXX [So that you won't goof as I did]
- # Note: Yes, I tried to use the -O & -OO flags here.
- # However the generated byte-codes were not portable.
- # So, we just stuck to un-optimized byte-codes,
- # which is still way better/faster than running
- # Python sources all the time.
- $python -m compileall $legacy -d '/' "$dst_dir" || {
- echo "python -m compileall err-ed"
- exit 1
- }
-
- # Delete source files and pyc [ un-optimized bytecode files ]
- # We may want to make this optimization thing configurable later, but not sure atm
- find "$dst_dir" -type f -name "*\.py" -exec rm -f {} \;
-
- delete_empty_dirs "$dst_dir"
-
- exit 0
|