From ec4235f11af71179f4e6653c8ea42d3debe5f0be Mon Sep 17 00:00:00 2001 From: giovannibajo Date: Fri, 20 Feb 2009 16:23:41 +0000 Subject: [PATCH] Document better how to build the bootloader under Linux and Mac git-svn-id: http://svn.pyinstaller.org/trunk@605 8dd32b29-ccff-0310-8a9a-9233e24343b1 --- doc/Manual.html | 167 ++++++++++++++++++++++-------------------- doc/source/Manual.rst | 18 ++++- 2 files changed, 103 insertions(+), 82 deletions(-) diff --git a/doc/Manual.html b/doc/Manual.html index 2c006cb..d75634c 100644 --- a/doc/Manual.html +++ b/doc/Manual.html @@ -3,7 +3,7 @@ - + PyInstaller Manual @@ -21,8 +21,8 @@ Contact: william@hpcf.upr.edu Revision: -282 -Source URL:svn://pyinstaller/branches/crypt/doc/source/Manual.rst +340 +Source URL:$HeadURL: http://svn.pyinstaller.python-hosting.com/trunk/doc/source/Manual.rst $ Copyright: This document has been placed in the public domain. @@ -144,16 +144,27 @@ to these scripts short (don't install in a deeply nested subdirectory).

other words, you will need a separate copy of PyInstaller for each Python version you wish to work with or you'll need to rerun Configure.py every time you switch the Python version).

-

Back to Top

+

Back to Top

Building the runtime executables

Note: Windows users can skip this step, because all of Python is contained in pythonXX.dll, and PyInstaller will use your pythonXX.dll.

-

On Linux the first thing to do is build the runtime executables.

-

Change to the /your/path/to/pyinstaller/ source/linux subdirectory. Run Make.py -[-n|-e] and then make. This will produce support/loader/run and -support/loader/run_d, which are the bootloaders.

+

On Linux the first thing to do is build the runtime executables. To do that, +you need to have some basic C/C++ development tools and the Python development +libraries. On Debian/Ubuntu systems, you can run the following lines +to install everything required:

+
+sudo apt-get install build-essential python-dev
+
+

Change to the /your/path/to/pyinstaller/ source/linux subdirectory. Run:

+
+pyinstaller$ cd source/linux
+pyinstaller/source/linux$ python Make.py  #[-n|-e]
+pyinstaller/source/linux$ make
+
+

This will produce support/loader/run and support/loader/run_d, +which are the bootloaders.

Bootloader

The bootloader (also known as stub in literature) is the small program @@ -176,7 +187,7 @@ directory: run_*.exe inprocsrvr_*.dll (bootloader for in-process COM servers). To rebuild this, you need to install Scons, and then just run scons from the /your/path/to/pyinstaller/ directory.

-

Back to Top

+

Back to Top

Configuring your PyInstaller setup

@@ -184,7 +195,7 @@ directory.

information into config.dat that would otherwise be recomputed every time. It can be rerun at any time if your configuration changes. It must be run before trying to build anything.

-

Back to Top

+

Back to Top

Create a spec file for your project

@@ -269,7 +280,7 @@ generated). If omitted, the basename of the (first) script is used. more complex projects, it should be regarded as a template. The spec file is actually Python code, and modifying it should be ease. See Spec Files for details.

-

Back to Top

+

Back to Top

Build your project

@@ -284,7 +295,7 @@ interested in. For a --onefile

In most cases, this will be all you have to do. If not, see When things go wrong and be sure to read the introduction to Spec Files.

-

Back to Top

+

Back to Top

Windows COM Server support

@@ -329,7 +340,7 @@ o = win32com.client.Dispatch(progid,

MakeCOMServer also assumes that your top level code (registration etc.) is "normal". If it's not, you will have to edit the generated script.

-

Back to Top

+

Back to Top

Building Optimized

@@ -346,7 +357,7 @@ exe = EXE(pyz, ...

See Spec Files for details.

-

Back to Top

+

Back to Top

A Note on using UPX

@@ -379,10 +390,10 @@ see the UPX docs for more information). So for now, at least, you can't use upx for CGI's executed by Apache. Otherwise, you can ignore the warnings in the UPX docs, since what PyInstaller opens is the executable Installer created, not the temporary upx-created executable.

-

Back to Top

+

Back to Top

-

A Note on --onefile

+

A Note on --onefile

A --onefile works by packing all the shared libs / dlls into the archive attached to the bootloader executable (or next to the executable in a non-elf configuration). When first started, it finds that it needs to extract these @@ -417,7 +428,7 @@ lookalike of one of the shared libraries during the hole between when the library is extracted and when it gets loaded by the execvp'd process. So maybe you shouldn't do setuid root programs using --onefile. In fact, we do not recomend the use of --onefile on setuid programs.

-

Back to Top

+

Back to Top

A Note on .egg files and setuptools

@@ -437,7 +448,7 @@ PyInstaller to see the files and make the package correctly. If you have already installed the modules, you can simply decompress them within a directory with the same name of the egg file (including also the extension).

Support for egg files is planned for a future release of PyInstaller.

-

Back to Top

+

Back to Top

@@ -462,7 +473,7 @@ given, extracted to stdout.
Q
Quit.
-

Back to Top

+

Back to Top

bindepend

@@ -495,14 +506,14 @@ how the data is displayed and the structure of the resource itself. So the easiest thing to do is find an executable that displays the kind of information you want, grab it's resource and edit it. Certainly easier than the Version resource wizard in VC++.

-

Back to Top

+

Back to Top

Analyzing Dependencies

You can interactively track down dependencies, including getting cross-references by using mf.py, documented in section mf.py: A modulefinder Replacement

-

Back to Top

+

Back to Top

@@ -541,7 +552,7 @@ and binaries. A EXE is built from the PYZ, the scripts and, in the case of a single-file deployment, the binaries. In a single-directory deployment, a directory is built containing a slim executable and the binaries.

-

Back to Top

+

Back to Top

TOC Class (Table of Contents)

@@ -689,7 +700,7 @@ the run executables understand are:

becomes possible to factor out common modules, and deploy a project containing multiple executables with minimal redundancy. You'll need some top level code in each executable to mount the common PYZ.

-

Back to Top

+

Back to Top

Target Subclasses

@@ -721,7 +732,7 @@ prepended. filtered. On Windows, a long list of MS dlls are excluded. On Linux/Unix, any shared lib in /lib or /usr/lib is excluded. -

Back to Top

+

Back to Top

PYZ

@@ -736,7 +747,7 @@ PYZ(toc, name=None, level=9)
level
The Zlib compression level to use. If 0, the zlib module is not required.
-

Back to Top

+

Back to Top

PKG

@@ -759,7 +770,7 @@ uses sensible values. If zlib is not available, no compression is used.
If 1, EXTENSIONs and BINARYs will be left out of the PKG, and forwarded to its container (usually a COLLECT).
-

Back to Top

+

Back to Top

EXE

@@ -801,7 +812,7 @@ an executable does not work, appe -

Back to Top

+

Back to Top

DLL

@@ -810,7 +821,7 @@ generalized. However, embedders can follow the same model to build a special purpose DLL so the Python support in their app is hidden. You will need to write your own dll, but thanks to Allan Green for refactoring the C code and making that a managable task.

-

Back to Top

+

Back to Top

COLLECT

@@ -828,7 +839,7 @@ COLLECT(*args, **kws) -

Back to Top

+

Back to Top

Tree

@@ -852,7 +863,7 @@ Tree(root, prefix=None, excludes=None)

Since a Tree is a TOC, you can also use the exclude technique described above in the section on TOCs.

-

Back to Top

+

Back to Top

@@ -888,7 +899,7 @@ Both exec and

Any problem detected here can be handled by hooking the analysis of the module. See Listing Hidden Imports below for how to do it.

-

Back to Top

+

Back to Top

Getting Debug Messages

@@ -896,7 +907,7 @@ See Listing Hidden Imports -

Back to Top

+

Back to Top

Getting Python's Verbose Imports

@@ -915,7 +926,7 @@ EXE(..., anal.scripts + [('v', '', 'OPTION')], ...)

These messages will always go to stdout, so you won't see them on Windows if console=0.

-

Back to Top

+

Back to Top

@@ -938,7 +949,7 @@ anal = Analysis(['somedir/myscript.py'], Makespec.py --paths=path/to/thisdir;path/to/thatdir ...

(on *nix, use : as the path separator).

-

Back to Top

+

Back to Top

Listing Hidden Imports

@@ -973,10 +984,10 @@ explicitly imported module1Hooks section).

If you successfully hook a publicly distributed module in this way, please send us the hook so we can make it available to others.

-

Back to Top

+

Back to Top

-

Extending a Package's __path__

+

Extending a Package's __path__

Python allows a package to extend the search path used to find modules and sub-packages through the __path__ mechanism. Normally, a package's __path__ has only one entry - the directory in which the __init__.py was found. But @@ -997,7 +1008,7 @@ for an example.

and only the analysis. That is, at runtime win32com.shell is resolved the same way as win32com.anythingelse, and win32com.__path__ knows nothing of ../win32comext.

Once in awhile, that's not enough.

-

Back to Top

+

Back to Top

Changing Runtime Behavior

@@ -1022,7 +1033,7 @@ my persistence scheme).

free to do almost anything. One provided hook sets things up so that win32com can generate modules at runtime (to disk), and the generated modules can be found in the win32com package.

-

Back to Top

+

Back to Top

Adapting to being "frozen"

@@ -1035,7 +1046,7 @@ probably want to look for it based on sys.frozen='dll').

For really advanced users, you can access the iu.ImportManager as sys.importManager. See iu.py for how you might make use of this fact.

-

Back to Top

+

Back to Top

Accessing Data Files

@@ -1063,7 +1074,7 @@ data = this.extract('mystuff')[1]

to get the contents as a binary string. See support/unpackTK.py for an advanced example (the TCL and TK lib files are in a PKG which is opened in place, and then extracted to the filesystem).

-

Back to Top

+

Back to Top

@@ -1074,7 +1085,7 @@ then extracted to the filesystem).

Pmw comes with a script named bundlepmw in the bin directory. If you follow the instructions in that script, you'll end up with a module named Pmw.py. Ensure that Builder finds that module and not the development package.

-

Back to Top

+

Back to Top

Win9xpopen

@@ -1082,7 +1093,7 @@ that Builder finds that module and not the development package.

need to distribute win9xpopen.exe with your app. On older Pythons with Win32all, this would apply to Win32pipe and win32popenWin9x.exe. (On yet older Pythons, no form of popen worked on Win9x).

-

Back to Top

+

Back to Top

Self-extracting executables

@@ -1094,7 +1105,7 @@ end and 'open' the CArchiveOn other platforms, the archive and the executable are separate, but the archive is named executable.pkg, and expected to be in the same directory. Other than that, the process is the same.

-

Back to Top

+

Back to Top

One Pass Execution

In a single directory deployment (--onedir, which is the default), all of the @@ -1109,7 +1120,7 @@ bootstraps the import hooks)

  • runs all the scripts which are at the top level of the archive
  • finalizes Python
    • -

    Back to Top

    +

    Back to Top

    Two Pass Execution

    @@ -1148,7 +1159,7 @@ installed from RPM, you need the Python-development RPM). It also overrides sys.path.

    In both cases, while one PyInstaller download can be used with any Python version, you need to have separate installations for each Python version.

    -

    Back to Top

    +

    Back to Top

    @@ -1164,10 +1175,10 @@ directly. This is a ZlibArchive -

    Back to Top

    +

    Back to Top

    -

    ZlibArchive

    +

    ZlibArchive

    A ZlibArchive contains compressed .pyc (or .pyo) files. The Table of Contents is a marshalled dictionary, with the key (the module's name as given in an import statement) associated with a seek position and length. Because it is @@ -1182,10 +1193,10 @@ entry was created from (the __fil compiled). On a user's box with no source installed, this is not terribly useful, but if they send you the traceback, at least you can make sense of it.

    ZlibArchiveImage

    -

    Back to Top

    +

    Back to Top

    -

    CArchive

    +

    CArchive

    A CArchive contains whatever you want to stuff into it. It's very much like a .zip file. They are easy to create in Python and unpack from C code. CArchives can be appended to other files (like ELF and COFF executables, for example). @@ -1201,7 +1212,7 @@ file. The name is null terminated. Compression is optional by member.

    CArchive as a .zip file, you don't need to worry about this. The type codes are used by the self-extracting executables.

    CArchiveImage

    -

    Back to Top

    +

    Back to Top

    @@ -1224,7 +1235,7 @@ words, any modifications to will have to be distributed under GPL.

    For updated information or clarification see our FAQ at PyInstaller home page: http://pyinstaller.hpcf.upr.edu

    -

    Back to Top

    +

    Back to Top

    Appendix

    @@ -1235,13 +1246,13 @@ insights of the internal workings of PyInstaller, and you do not need this information unless you plan to work on PyInstaller itself.

    -

    mf.py: A Modulefinder Replacement

    +

    mf.py: A Modulefinder Replacement

    Module mf is modelled after iu.

    It also uses ImportDirectors and Owners to partition the import name space. Except for the fact that these return Module instances instead of real module objects, they are identical.

    Instead of an ImportManager, mf has an ImportTracker managing things.

    -

    Back to Top

    +

    Back to Top

    ImportTracker

    ImportTracker can be called in two ways: analyze_one(name, importername=None) @@ -1251,10 +1262,10 @@ cause to appear in sys.modules -

    Back to Top

    +

    Back to Top

    -

    analyze_one()

    +

    analyze_one()

    When a name is imported, there are structural and dynamic effects. The dynamic effects are due to the execution of the top-level code in the module (or modules) that get imported. The structural effects have to do with whether the @@ -1266,7 +1277,7 @@ dynamic effects. For example, ana or ["A.B", "A.B.C"] depending on whether the import turns out to be relative or absolute. In addition, ImportTracker's modules dict will have Module instances for them.

    -

    Back to Top

    +

    Back to Top

    Module Classes

    @@ -1281,7 +1292,7 @@ it's top-level code executed. That top-level code can do various things so that when the import of B.C finally occurs, something completely different happens (from what a structural analysis would predict). But mf can handle this through it's hooks mechanism.

    -

    Back to Top

    +

    Back to Top

    code scanning

    @@ -1295,7 +1306,7 @@ and can issue warnings when they're found.

    import. It recognizes when imports are found at the top-level, and when they are found inside definitions (deferred imports). Within that, it also tracks whether the import is inside a condition (conditional imports).

    -

    Back to Top

    +

    Back to Top

    Hooks

    @@ -1325,7 +1336,7 @@ trace through PyXML-using code, and I can't believe that it's any easier for the poor programmer using that package). The hook(mod) (if it exists) is called before looking at the others - that way it can, for example, test sys.version and adjust what's in hiddenimports.

    -

    Back to Top

    +

    Back to Top

    Warnings

    @@ -1336,7 +1347,7 @@ during the structural phase, and during the code scan.

    Note that by using a hook module, you can silence some particularly tiresome warnings, but not all of them.

    -

    Back to Top

    +

    Back to Top

    Cross Reference

    @@ -1345,7 +1356,7 @@ cross reference by using getxref( is (modulename, importers), where importers is a list of the (fully qualified) names of the modules importing modulename. Both the returned list and the importers list are sorted.

    -

    Back to Top

    +

    Back to Top

    Usage

    @@ -1393,11 +1404,11 @@ W: delayed eval hack detected at line 418 >>> -

    Back to Top

    +

    Back to Top

    -

    iu.py: An imputil Replacement

    +

    iu.py: An imputil Replacement

    Module iu grows out of the pioneering work that Greg Stein did with imputil (actually, it includes some verbatim imputil code, but since Greg didn't copyright it, we won't mention it). Both modules can take over Python's @@ -1409,9 +1420,9 @@ builtin import and ease writing of at least certain kinds of import hooks.

    There is an ImportManager which provides the replacement for builtin import and hides all the semantic complexities of a Python import request from it's delegates.

    -

    Back to Top

    +

    Back to Top

    -

    ImportManager

    +

    ImportManager

    ImportManager formalizes the concept of a metapath. This concept implicitly exists in native Python in that builtins and frozen modules are searched before sys.path, (on Windows there's also a search of the registry while on @@ -1425,18 +1436,18 @@ metapath until one succeeds.

    It's up to the ImportManager to decide if an import is relative or absolute; to see if the module has already been imported; to keep sys.modules up to date; to handle the fromlist and return the correct module object.

    -

    Back to Top

    +

    Back to Top

    -

    ImportDirector

    +

    ImportDirector

    An ImportDirector just needs to respond to getmod(name) by returning a module object or None. As you will see, an ImportDirector can consider name to be atomic - it has no need to examine name to see if it is dotted.

    To see how this works, we need to examine the PathImportDirector.

    -

    Back to Top

    +

    Back to Top

    -

    PathImportDirector

    +

    PathImportDirector

    The PathImportDirector subclass manages a list of names - most notably, sys.path. To do so, it maintains a shadowpath - a dictionary mapping the names on its pathlist (eg, sys.path) to their associated Owners. (It could do this @@ -1444,10 +1455,10 @@ directly, but the assumption that sys.path is occupied solely by strings seems ineradicable.) Owners of the appropriate kind are created as needed (if all your imports are satisfied by the first two elements of sys.path, the PathImportDirector's shadowpath will only have two entries).

    -

    Back to Top

    +

    Back to Top

    -

    Owner

    +

    Owner

    An Owner is much like an ImportDirector but manages a much more concrete piece of turf. For example, a DirOwner manages one directory. Since there are no other officially recognized filesystem-like namespaces for importing, that's @@ -1462,7 +1473,7 @@ consisting of Owners. namespace.

    The rest of the import namespace is covered by treelets, each rooted in a package module (an __init__.py).

    -

    Back to Top

    +

    Back to Top

    Packages

    @@ -1480,7 +1491,7 @@ module name, not the package name or the fully qualified name. And that's exactly what it gets. (In a flat namespace - like most archives - it is perfectly easy to route the request back up the package tree to the archive Owner, qualifying the name at each step.)

    -

    Back to Top

    +

    Back to Top

    Possibilities

    @@ -1495,7 +1506,7 @@ lines of code all told for my own archive format, including initializing a pack age with it's __subimporter__).

    Once the new Owner class is registered with iu, you can put a zip file on sys.path. A package could even put a zip file on its __path__.

    -

    Back to Top

    +

    Back to Top

    Compatibility

    @@ -1504,7 +1515,7 @@ over a dozen import hacks from manipulations of sys.modules with a different one. Emulation of Python's native import is nearly exact, including the names recorded in sys.modules and module attributes (packages imported through iu have an extra attribute - __importsub__).

    -

    Back to Top

    +

    Back to Top

    Performance

    @@ -1513,7 +1524,7 @@ nearly exact, including the names recorded in ImportManager's metapath.

    -

    Back to Top

    +

    Back to Top

    Limitations

    @@ -1547,7 +1558,7 @@ easy to implement.

    >>> -

    Back to Top

    +

    Back to Top

    @@ -1555,7 +1566,7 @@ easy to implement.

    View document source. -Generated on: 2007-11-27 16:36 UTC. +Generated on: 2009-02-20 16:22 UTC. Generated by Docutils from reStructuredText source.
    diff --git a/doc/source/Manual.rst b/doc/source/Manual.rst index 49c3ad9..3c87f93 100644 --- a/doc/source/Manual.rst +++ b/doc/source/Manual.rst @@ -37,11 +37,21 @@ Building the runtime executables *Note:* Windows users can skip this step, because all of Python is contained in pythonXX.dll, and |PyInstaller| will use your pythonXX.dll. -On Linux the first thing to do is build the runtime executables. +On Linux the first thing to do is build the runtime executables. To do that, +you need to have some basic C/C++ development tools and the Python development +libraries. On Debian/Ubuntu systems, you can run the following lines +to install everything required:: -Change to the |install_path| ``source/linux`` subdirectory. Run ``Make.py -[-n|-e]`` and then make. This will produce ``support/loader/run`` and -``support/loader/run_d``, which are the bootloaders. + sudo apt-get install build-essential python-dev + +Change to the |install_path| ``source/linux`` subdirectory. Run:: + + pyinstaller$ cd source/linux + pyinstaller/source/linux$ python Make.py #[-n|-e] + pyinstaller/source/linux$ make + +This will produce ``support/loader/run`` and ``support/loader/run_d``, +which are the bootloaders. .. sidebar:: Bootloader