-
-
- ArchiveViewer -
- bindepend -
- GrabVersion (Windows) -
- Analyzing Dependencies +
- PyInstaller Utilities -
- Spec Files
-
-
- Introduction -
- TOC Class (Table of Contents) -
- Target Subclasses
-
-
- Analysis -
- PYZ -
- PKG -
- EXE -
- DLL -
- COLLECT -
- Tree +
- Spec Files -
- When Things Go Wrong
-
-
- Finding out What Went Wrong
-
-
- Buildtime Warnings -
- Getting Debug Messages -
- Getting Python's Verbose Imports +
- When Things Go Wrong
-
+
- Finding out What Went Wrong -
- Helping Installer Find Modules -
- Miscellaneous
-
-
- Pmw -- Python Mega Widgets -
- Win9xpopen -
- Self-extracting executables
-
-
- One Pass Execution -
- Two Pass Execution +
- Miscellaneous -
- PyInstaller Archives
-
-
- Archives Introduction -
- ZlibArchive -
- CArchive +
- PyInstaller Archives -
- License -
- Appendix
-
-
- mf.py: A Modulefinder Replacement
-
-
- ImportTracker -
- analyze_one() -
- Module Classes -
- code scanning -
- Hooks -
- Warnings -
- Cross Reference -
- Usage +
- License +
- Appendix
-
+
- mf.py: A Modulefinder Replacement -
- iu.py: An imputil Replacement
-
-
- ImportManager -
- ImportDirector -
- PathImportDirector -
- Owner -
- Packages -
- Possibilities -
- Compatibility -
- Performance -
- Limitations -
- Usage +
- iu.py: An imputil Replacement
---onefile and Python 2.4 for Windows (important)
-Currently, there is an issue when using --onefile with Python 2.4: the -resulting executable will depend on MSVCR71.DLL. This is a standard -Microsoft library which was not present on older Windows (like Win9x), so -you are forced to ship it with your application if you need compatibility -with those operating systems. We plan to fix this issue in a future version -of PyInstaller (consult our Roadmap for more information).
- --PyInstaller Utilities
+PyInstaller Utilities
-ArchiveViewer
+ArchiveViewer
python ArchiveViewer.py <archivefile>
@@ -445,10 +434,10 @@ given, extracted to stdout.- Q
- Quit.
- +-bindepend
+bindepend
python bindepend.py <executable_or_dynamic_library>
@@ -459,7 +448,7 @@ follow the chain of dependencies of binary extensions and make sure that all of them get included in the final package.-GrabVersion (Windows)
+GrabVersion (Windows)
python GrabVersion.py <executable_with_version_resource>
@@ -478,20 +467,20 @@ how the data is displayed and the structure of the resource itself. So the easiest thing to do is find an executable that displays the kind of information you want, grab it's resource and edit it. Certainly easier than the Version resource wizard in VC++. - +-Analyzing Dependencies
+Analyzing Dependencies
You can interactively track down dependencies, including getting cross-references by using mf.py, documented in section mf.py: A modulefinder Replacement
- +-Spec Files
+Spec Files
-Introduction
+Introduction
Spec files are in Python syntax. They are evaluated by Build.py. A simplistic spec file might look like this:
@@ -524,10 +513,10 @@ and binaries. A EXE is built from the PYZ, the scripts and, in the case of a single-file deployment, the binaries. In a single-directory deployment, a directory is built containing a slim executable and the binaries. - +
-TOC Class (Table of Contents)
+TOC Class (Table of Contents)
Before you can do much with a spec file, you need to understand the TOC (Table Of Contents) class.
A TOC appears to be a list of tuples of the form (name, path, typecode). @@ -672,12 +661,12 @@ the run executables understand are:
becomes possible to factor out common modules, and deploy a project containing multiple executables with minimal redundancy. You'll need some top level code in each executable to mount the common PYZ. - +-Target Subclasses
+Target Subclasses
-Analysis
+Analysis
Analysis(scripts, pathex=None, hookspath=None, excludes=None)
@@ -704,10 +693,10 @@ prepended. filtered. On Windows, a long list of MS dlls are excluded. On Linux/Unix, any shared lib in /lib or /usr/lib is excluded. - +-PYZ
+PYZ
PYZ(toc, name=None, level=9)
@@ -719,10 +708,10 @@ PYZ(toc, name=None, level=9)- level
- The Zlib compression level to use. If 0, the zlib module is not required.
- +-PKG
+PKG
Generally, you will not need to create your own PKGs, as the EXE will do it for you. This is one way to include read-only data in a single-file deployment, however. A single-file deployment including TK support will use this technique.
@@ -742,10 +731,10 @@ uses sensible values. If zlib is not available, no compression is used.- If 1, EXTENSIONs and BINARYs will be left out of the PKG, and forwarded to its container (usually a COLLECT).
- +-EXE
+EXE
EXE(*args, **kws)
@@ -782,19 +771,19 @@ and one for non-ELF platforms (where the run executable is simply renamed, and expects a exename.pkg in the same directory). Which class becomes available as EXE is determined by a flag in config.dat. This flag is set to non-ELF when using Make.py -n. - +-DLL
+DLL
On Windows, this provides support for doing in-process COM servers. It is not generalized. However, embedders can follow the same model to build a special purpose DLL so the Python support in their app is hidden. You will need to write your own dll, but thanks to Allan Green for refactoring the C code and making that a managable task.
- +-When Things Go Wrong
+When Things Go Wrong
-Finding out What Went Wrong
+Finding out What Went Wrong
-Buildtime Warnings
+Buildtime Warnings
When an Analysis step runs, it produces a warnings file (named warnproject.txt) in the spec file's directory. Generally, most of these warnings are harmless. For example, os.py (which is cross-platform) works by figuring out what @@ -869,18 +858,18 @@ Both exec and
Any problem detected here can be handled by hooking the analysis of the module. See Listing Hidden Imports below for how to do it.
- +-Getting Debug Messages
+Getting Debug Messages
Setting debug=1 on an EXE will cause the executable to put out progress messages (for console apps, these go to stdout; for Windows apps, these show as MessageBoxes). This can be useful if you are doing complex packaging, or your app doesn't seem to be starting, or just to learn how the runtime works.
- +-Getting Python's Verbose Imports
+Getting Python's Verbose Imports
You can also pass a -v (verbose imports) flag to the embedded Python. This can be extremely useful. I usually try it even on apparently working apps, just to make sure that I'm always getting my copies of the modules and no import has @@ -896,13 +885,13 @@ EXE(..., anal.scripts + [('v', '', 'OPTION')], ...)
These messages will always go to stdout, so you won't see them on Windows if console=0.
- +-Helping Installer Find Modules
+Helping Installer Find Modules
-Extending the Path
+Extending the Path
When the analysis phase cannot find needed modules, it may be that the code is manipulating sys.path. The easiest thing to do in this case is tell Analysis about the new directory through the second arg to the constructor:
@@ -919,10 +908,10 @@ anal = Analysis(['somedir/myscript.py'], Makespec.py --paths=path/to/thisdir;path/to/thatdir ...(on *nix, use : as the path separator).
- +-+
Hidden imports are fairly common. These can occur when the code is using __import__ (or, perhaps exec or eval), in which case you will see a warning in the warnproject.txt file. They can also occur when an extension module uses the @@ -954,10 +943,10 @@ explicitly imported module1Hooks section).
If you successfully hook a publicly distributed module in this way, please send us the hook so we can make it available to others.
- +-Extending a Package's __path__
+Extending a Package's __path__
Python allows a package to extend the search path used to find modules and sub-packages through the __path__ mechanism. Normally, a package's __path__ has only one entry - the directory in which the __init__.py was found. But @@ -978,10 +967,10 @@ for an example.
and only the analysis. That is, at runtime win32com.shell is resolved the same way as win32com.anythingelse, and win32com.__path__ knows nothing of ../win32comext.Once in awhile, that's not enough.
- +-Changing Runtime Behavior
+Changing Runtime Behavior
More bizarre situations can be accomodated with runtime hooks. These are small scripts that manipulate the environment before your main script runs, effectively providing additional top-level code to your script.
@@ -1003,10 +992,10 @@ my persistence scheme). free to do almost anything. One provided hook sets things up so that win32com can generate modules at runtime (to disk), and the generated modules can be found in the win32com package. - +-Adapting to being "frozen"
+Adapting to being "frozen"
In most sophisticated apps, it becomes necessary to figure out (at runtime) whether you're running "live" or "frozen". For example, you might have a configuration file that (running "live") you locate based on a module's @@ -1016,10 +1005,10 @@ probably want to look for it based on ).
For really advanced users, you can access the iu.ImportManager as sys.importManager. See iu.py for how you might make use of this fact.
- +-Accessing Data Files
+Accessing Data Files
In a --onedir distribution, this is easy: pass a list of your data files (in TOC format) to the COLLECT, and they will show up in the distribution directory tree. The name in the (name, path, 'DATA') tuple can be a relative @@ -1044,29 +1033,29 @@ data = this.extract('mystuff')[1]
to get the contents as a binary string. See support/unpackTK.py for an advanced example (the TCL and TK lib files are in a PKG which is opened in place, and then extracted to the filesystem).
- +-Miscellaneous
+Miscellaneous
-Pmw -- Python Mega Widgets
+Pmw -- Python Mega Widgets
Pmw comes with a script named bundlepmw in the bin directory. If you follow the instructions in that script, you'll end up with a module named Pmw.py. Ensure that Builder finds that module and not the development package.
- +-Win9xpopen
+Win9xpopen
If you're using popen on Windows and want the code to work on Win9x, you'll need to distribute win9xpopen.exe with your app. On older Pythons with Win32all, this would apply to Win32pipe and win32popenWin9x.exe. (On yet older Pythons, no form of popen worked on Win9x).
- +-Self-extracting executables
+Self-extracting executables
The ELF executable format (Windows, Linux and some others) allows arbitrary data to be concatenated to the end of the executable without disturbing its functionality. For this reason, a CArchive's Table of Contents is at the end of @@ -1075,9 +1064,9 @@ end and 'open' the CArchiveOn other platforms, the archive and the executable are separate, but the archive is named executable.pkg, and expected to be in the same directory. Other than that, the process is the same.
- +-One Pass Execution
+One Pass Execution
In a single directory deployment (--onedir, which is the default), all of the binaries are already in the file system. In that case, the embedding app:
-
@@ -1090,10 +1079,10 @@ bootstraps the import hooks)
- runs all the scripts which are at the top level of the archive
- finalizes Python
-Two Pass Execution
+Two Pass Execution
There are a couple situations which require two passes:
- a --onefile deployment (on Windows, the files can't be cleaned up afterwards
@@ -1129,14 +1118,14 @@ installed from RPM, you need the Python-development RPM). It also overrides
sys.path.
In both cases, while one PyInstaller download can be used with any Python version, you need to have separate installations for each Python version.
- +
-PyInstaller Archives
+PyInstaller Archives
-Archives Introduction
+Archives Introduction
You know what an archive is: a .tar file, a .jar file, a .zip file. Two kinds of archives are used here. One is equivalent to a Java .jar file - it allows Python modules to be stored efficiently and, (with some import hooks) imported @@ -1145,10 +1134,10 @@ directly. This is a ZlibArchive - +
-ZlibArchive
+ZlibArchive
A ZlibArchive contains compressed .pyc (or .pyo) files. The Table of Contents is a marshalled dictionary, with the key (the module's name as given in an import statement) associated with a seek position and length. Because it is @@ -1163,10 +1152,10 @@ entry was created from (the __fil compiled). On a user's box with no source installed, this is not terribly useful, but if they send you the traceback, at least you can make sense of it.
- +
-CArchive
+CArchive
A CArchive contains whatever you want to stuff into it. It's very much like a .zip file. They are easy to create in Python and unpack from C code. CArchives can be appended to other files (like ELF and COFF executables, for example). @@ -1182,11 +1171,11 @@ file. The name is null terminated. Compression is optional by member.
CArchive as a .zip file, you don't need to worry about this. The type codes are used by the self-extracting executables.
- +
-License
+License
PyInstaller is mainly distributed under the GPL License but it has an exception such that you can use it to compile commercial products.
@@ -1205,10 +1194,10 @@ words, any modifications to will have to be distributed under GPL.For updated information or clarification see our FAQ at PyInstaller home page: http://pyinstaller.hpcf.upr.edu
- +-@@ -1536,7 +1525,7 @@ easy to implement. diff --git a/doc/source/Manual.rst b/doc/source/Manual.rst index 24179ec..5d80477 100644 --- a/doc/source/Manual.rst +++ b/doc/source/Manual.rst @@ -328,19 +328,6 @@ recomend the use of --onefile on setuid programs.** |GOBACK| -``--onefile`` and Python 2.4 for Windows (**important**) --------------------------------------------------------- - -Currently, there is an issue when using ``--onefile`` with Python 2.4: the -resulting executable will depend on ``MSVCR71.DLL``. This is a standard -Microsoft library which was not present on older Windows (like Win9x), so -you are forced to ship it with your application if you need compatibility -with those operating systems. We plan to fix this issue in a future version -of |PyInstaller| (consult our Roadmap_ for more information). - -|GOBACK| - - PyInstaller Utilities +++++++++++++++++++++ diff --git a/source/common/launch.c b/source/common/launch.c index c101e00..4088b4a 100644 --- a/source/common/launch.c +++ b/source/common/launch.c @@ -63,7 +63,7 @@ DECLPROC(PyObject_SetAttrString); DECLPROC(PyList_New); DECLPROC(PyList_Append); DECLPROC(Py_BuildValue); -DECLPROC(PyFile_FromFile); +DECLPROC(PyFile_FromString); DECLPROC(PyObject_CallFunction); DECLPROC(PyModule_GetDict); DECLPROC(PyDict_GetItemString); @@ -273,7 +273,7 @@ int mapNames(HMODULE dll) GETPROC(dll, PyList_New); GETPROC(dll, PyList_Append); GETPROC(dll, Py_BuildValue); - GETPROC(dll, PyFile_FromFile); + GETPROC(dll, PyFile_FromString); GETPROC(dll, PyObject_CallFunction); GETPROC(dll, PyModule_GetDict); GETPROC(dll, PyDict_GetItemString); @@ -570,6 +570,8 @@ int importModules() TOC *ptoc; PyObject *co; PyObject *mod; + PyObject *res; + char buf[32]; VS("importing modules from CArchive\n"); @@ -581,8 +583,17 @@ int importModules() marshaldict = PyModule_GetDict(marshal); loadfunc = PyDict_GetItemString(marshaldict, "load"); - /* Make a Python file object from f_fp */ - pyfile = PyFile_FromFile(f_fp, f_archivename, "rb+", 0); + /* Reopen the archive as a Python file. We cannot use PyFile_FromFile + * because that would require this boot-loader and Python DLL to share + * the same libc, while they purposely don't. + */ + fclose(f_fp); + pyfile = PyFile_FromString(f_archivename, "rb"); + if (PyErr_Occurred()) + { + PyErr_Print(); + return -1; + } /* Iterate through toc looking for module entries (type 'm') * this is normally just bootstrap stuff (archive and iu) @@ -593,8 +604,11 @@ int importModules() { VS(ptoc->name); VS("\n"); + /* Go to start of Python module (start + 8) and load the code object */ - fseek(f_fp, f_pkgstart + ntohl(ptoc->pos) + 8, SEEK_SET); + res = PyObject_CallMethod(pyfile, "seek", "(ii)", f_pkgstart + ntohl(ptoc->pos) + 8, 0); + Py_XDECREF(res); + co = PyObject_CallFunction(loadfunc, "O", pyfile); mod = PyImport_ExecCodeModule(ptoc->name, co); @@ -602,19 +616,28 @@ int importModules() if (mod == NULL) { FATALERROR("mod is NULL - "); FATALERROR(ptoc->name); - //return -1; } if (PyErr_Occurred()) { PyErr_Print(); PyErr_Clear(); - //FATALERROR("PyErr loading mod - "); - //FATALERROR(ptoc->name); - //return -1; } } ptoc = incrementTocPtr(ptoc); } + + /* Close the file and release the object. */ + res = PyObject_CallMethod(pyfile, "close", "()"); + Py_XDECREF(res); + Py_DECREF(pyfile); + + /* After closing the python file, we can reopen it as normal file. */ + f_fp = fopen(f_archivename, "rb"); + if (f_fp == NULL) { + VS("Cannot reopen archive: "); + VS(f_archivename); + VS("\n"); + } return 0; } diff --git a/source/common/launch.h b/source/common/launch.h index fa1d918..1316e91 100644 --- a/source/common/launch.h +++ b/source/common/launch.h @@ -136,7 +136,7 @@ EXTDECLPROC(int, PyObject_SetAttrString, (PyObject *, char *, PyObject *)); EXTDECLPROC(PyObject *, PyList_New, (int)); EXTDECLPROC(int, PyList_Append, (PyObject *, PyObject *)); EXTDECLPROC(PyObject *, Py_BuildValue, (char *, ...)); -EXTDECLPROC(PyObject *, PyFile_FromFile, (FILE *, char *, char *, int)); +EXTDECLPROC(PyObject *, PyFile_FromString, (char *, char *)); EXTDECLPROC(PyObject *, PyObject_CallFunction, (PyObject *, char *, ...)); EXTDECLPROC(PyObject *, PyModule_GetDict, (PyObject *)); EXTDECLPROC(PyObject *, PyDict_GetItemString, (PyObject *, char *));Appendix
+Appendix
-mf.py: A Modulefinder Replacement
+mf.py: A Modulefinder Replacement
Module mf is modelled after iu.
It also uses ImportDirectors and Owners to partition the import name space. Except for the fact that these return Module instances instead of real module objects, they are identical.
Instead of an ImportManager, mf has an ImportTracker managing things.
- +-ImportTracker
+ImportTracker
ImportTracker can be called in two ways: analyze_one(name, importername=None) or analyze_r(name, importername=None). The second method does what modulefinder does - it recursively finds all the module names that importing name would @@ -1232,10 +1221,10 @@ cause to appear in sys.modules - +
-analyze_one()
+analyze_one()
When a name is imported, there are structural and dynamic effects. The dynamic effects are due to the execution of the top-level code in the module (or modules) that get imported. The structural effects have to do with whether the @@ -1247,10 +1236,10 @@ dynamic effects. For example, ana or ["A.B", "A.B.C"] depending on whether the import turns out to be relative or absolute. In addition, ImportTracker's modules dict will have Module instances for them.
- +-Module Classes
+Module Classes
There are Module subclasses for builtins, extensions, packages and (normal) modules. Besides the normal module object attributes, they have an attribute imports. For packages and normal modules, imports is a list populated by @@ -1262,10 +1251,10 @@ it's top-level code executed. That top-level code can do various things so that when the import of B.C finally occurs, something completely different happens (from what a structural analysis would predict). But mf can handle this through it's hooks mechanism.
- +-code scanning
+code scanning
Like modulefinder, mf scans the byte code of a module, looking for imports. In addition, mf will pick out a module's __all__ attribute, if it is built as a list of constant names. This means that if a package declares an __all__ list @@ -1276,10 +1265,10 @@ and can issue warnings when they're found.
import. It recognizes when imports are found at the top-level, and when they are found inside definitions (deferred imports). Within that, it also tracks whether the import is inside a condition (conditional imports). - +-Hooks
+Hooks
In modulefinder, scanning the code takes the place of executing the code object. mf goes further and allows a module to be hooked (after it has been scanned, but before analyze_one is done with it). A hook is a module named @@ -1306,10 +1295,10 @@ trace through PyXML-using code, and I can't believe that it's any easier for the poor programmer using that package). The hook(mod) (if it exists) is called before looking at the others - that way it can, for example, test sys.version and adjust what's in hiddenimports.
- +-Warnings
+Warnings
ImportTracker has a getwarnings() method that returns all the warnings accumulated by the instance, and by the Module instances in its modules dict. Generally, it is ImportTracker who will accumulate the warnings generated @@ -1317,19 +1306,19 @@ during the structural phase, and during the code scan.
Note that by using a hook module, you can silence some particularly tiresome warnings, but not all of them.
- +-Cross Reference
+Cross Reference
Once a full analysis (that is, an analyze_r call) has been done, you can get a cross reference by using getxref(). This returns a list of tuples. Each tuple is (modulename, importers), where importers is a list of the (fully qualified) names of the modules importing modulename. Both the returned list and the importers list are sorted.
- +-iu.py: An imputil Replacement
+iu.py: An imputil Replacement
Module iu grows out of the pioneering work that Greg Stein did with imputil (actually, it includes some verbatim imputil code, but since Greg didn't copyright it, we won't mention it). Both modules can take over Python's @@ -1390,9 +1379,9 @@ builtin import and ease writing of at least certain kinds of import hooks.
There is an ImportManager which provides the replacement for builtin import and hides all the semantic complexities of a Python import request from it's delegates.
- +-ImportManager
+ImportManager
ImportManager formalizes the concept of a metapath. This concept implicitly exists in native Python in that builtins and frozen modules are searched before sys.path, (on Windows there's also a search of the registry while on @@ -1406,18 +1395,18 @@ metapath until one succeeds.
It's up to the ImportManager to decide if an import is relative or absolute; to see if the module has already been imported; to keep sys.modules up to date; to handle the fromlist and return the correct module object. - +-ImportDirector
+ImportDirector
An ImportDirector just needs to respond to getmod(name) by returning a module object or None. As you will see, an ImportDirector can consider name to be atomic - it has no need to examine name to see if it is dotted.
To see how this works, we need to examine the PathImportDirector.
- +-PathImportDirector
+PathImportDirector
The PathImportDirector subclass manages a list of names - most notably, sys.path. To do so, it maintains a shadowpath - a dictionary mapping the names on its pathlist (eg, sys.path) to their associated Owners. (It could do this @@ -1425,10 +1414,10 @@ directly, but the assumption that sys.path is occupied solely by strings seems ineradicable.) Owners of the appropriate kind are created as needed (if all your imports are satisfied by the first two elements of sys.path, the PathImportDirector's shadowpath will only have two entries).
- +-Owner
+Owner
An Owner is much like an ImportDirector but manages a much more concrete piece of turf. For example, a DirOwner manages one directory. Since there are no other officially recognized filesystem-like namespaces for importing, that's @@ -1443,10 +1432,10 @@ consisting of Owners. namespace.
The rest of the import namespace is covered by treelets, each rooted in a package module (an __init__.py).
- +-Packages
+Packages
To make this work, Owners need to recognize when a module is a package. For a DirOwner, this means that name is a subdirectory which contains an __init__.py. The __init__ module is loaded and its __path__ is initialized with the @@ -1461,10 +1450,10 @@ module name, not the package name or the fully qualified name. And that's exactly what it gets. (In a flat namespace - like most archives - it is perfectly easy to route the request back up the package tree to the archive Owner, qualifying the name at each step.)
- +-Possibilities
+Possibilities
Let's say we want to import from zip files. So, we subclass Owner. The __init__ method should take a filename, and raise a ValueError if the file is not an acceptable .zip file, (when a new name is encountered on sys.path or a @@ -1476,28 +1465,28 @@ lines of code all told for my own archive format, including initializing a pack age with it's __subimporter__).
Once the new Owner class is registered with iu, you can put a zip file on sys.path. A package could even put a zip file on its __path__.
- +-Compatibility
+Compatibility
This code has been tested with the PyXML, mxBase and Win32 packages, covering over a dozen import hacks from manipulations of __path__ to replacing a module in sys.modules with a different one. Emulation of Python's native import is nearly exact, including the names recorded in sys.modules and module attributes (packages imported through iu have an extra attribute - __importsub__).
- +-Performance
+Performance
In most cases, iu is slower than builtin import (by 15 to 20%) but faster than imputil (by 15 to 20%). By inserting archives at the front of sys.path containing the standard lib and the package being tested, this can be reduced to 5 to 10% slower (or, on my 1.52 box, 10% faster!) than builtin import. A bit more can be shaved off by manipulating the ImportManager's metapath.
- +-Limitations
+Limitations
This module makes no attempt to facilitate policy import hacks. It is easy to implement certain kinds of policies within a particular domain, but fundamentally iu works by dividing up the import namespace into independent @@ -1514,7 +1503,7 @@ syntax for those should be, but _ easy to implement.
- mf.py: A Modulefinder Replacement
- Finding out What Went Wrong