mirror of
https://github.com/kennethreitz/tablib.git
synced 2026-06-05 23:10:17 +00:00
a21f8187f8
Squashing two squashes. Adding DBF support Adding the DBFpy python package The DBFpy package provides basic dbf support for python. Still need to write an interface format file for tablib. Adding DBF format and imports in compat.py Adding DBF format to formats.__init__ DBF format had not been committed to formats.__init__, so I’m adding it. Adding a dbf import test Adding at test to check whether a DBF can be created properly and compare it against a regression binary string. Adding an import_set test (and renaming another) Adding an import_set test that conforms with the other import_set tests for other formats. I’m also adding an export_set function. Fixing system site-packages import Importing dbfpy from tab lib.packages instead of system site packages. Fixing a syntaxError in dbfpy/dbfnew.py Fixing an issue with ending field definitions DBFPY, when writing a DBF, terminates the field definitions with a newline character. When importing a DBF from a stream, however, DBFPY was looking only for the \x0D character rather than the newline. Now we consider both cases. Adding a test for dbf format detection Adding DBF filetype detection tests Adding tests for YAML, JSON, TSV, CSV using the DBF detection function. Handling extra exceptions in dbf detection Adding exception handling for struct.error, an exception that DBFPY raises when trying to unpack a TSV table. Since it’s not a DBF file, we know it’s not a DBF and return False. Fixing an issue with the DBF set exporting test The DBF set export test needed a bit enabled (probably the writeable bit?) before the test would match the regression output. Updating dbf interface Updating the int/float class/type checking in the dbf format file. This allows for python2 and python3 compatibility. Tweaking dbfpy to work with python3 Altering a couple of imports. Updating dbf tests for binary data compatibility Making regression strings binary and improving debug messages for dbf assertion errors. Improving file handling for python 2 and 3 Updating DBF file handling for both python 2 and 3 in the _dbf interface. Adding a (seemingly) functional dbfpy for python3 I’ve made dbfpy python3 compatible! Tests appear to pass. A significant change was made to the format detection test whereby I made the input string a binary (bytes) string. If the string is not a bytes string by the time we try to detect the format, we try to decode the string as utf-8 (which admittedly might not be the safest thing to do) and try to decode anyways. Updating imports for tablib dbf interface Now importing python2 or python3 versions as appropriate. Updating dbf package references in compat.py Cleaning up debugging print statements Updating stream handling in dbf interface Factoring the open() call out of the py3 conditional and removing the temp file before returning the stream value. Adding dbfpy3 init.py I had apparently missed the dbfpy3 init file when committing dbfpy3. Adding dbfpy and dbfpy3 to setup.py's package list Switching test order of formats Putting dbf format testing ahead of TSV. In some of my tests with numeric DBF files, I encountered an issue where the ASCII horizontal tab character (0x09) would appear in a numeric DBF. Because of the order of tabular format imports, though, format detection would recognize it as a TSV and not as a DBF. Adding my name to AUTHORS. Adding a DBF property to tab lib core Documentation includes examples on how to explicitly load a DBF straight from a file and how to load a DBF from a binary string. Also, how to write the binary data to a file. Adding DBF format notes to README Adding exclamation point to DBF section title Matching formatting of XLS section Updating setup.py to match current dev state Setup.py had been updated since I forked the tablib repo, so I’m updating setup.py to match its current structure while still maintaining DBF compatibility. Fixed callable collumn test the test was sending a list instead of a function CORE CONTRIBUTORS 🍰 @iurisilvio v0.10.0 WHEELS 3.3, 3.4 makefile for WHEELS v0.10.0 history ALL Separate py2 and py3 packages to avoid installation errors. Fix #151 Running travis and tox with python 3.4. Adding DBF support Adding the DBFpy python package The DBFpy package provides basic dbf support for python. Still need to write an interface format file for tablib. Adding DBF format and imports in compat.py Adding DBF format to formats.__init__ DBF format had not been committed to formats.__init__, so I’m adding it. Adding a dbf import test Adding at test to check whether a DBF can be created properly and compare it against a regression binary string. Adding an import_set test (and renaming another) Adding an import_set test that conforms with the other import_set tests for other formats. I’m also adding an export_set function. Fixing system site-packages import Importing dbfpy from tab lib.packages instead of system site packages. Fixing a syntaxError in dbfpy/dbfnew.py Fixing an issue with ending field definitions DBFPY, when writing a DBF, terminates the field definitions with a newline character. When importing a DBF from a stream, however, DBFPY was looking only for the \x0D character rather than the newline. Now we consider both cases. Adding a test for dbf format detection Adding DBF filetype detection tests Adding tests for YAML, JSON, TSV, CSV using the DBF detection function. Handling extra exceptions in dbf detection Adding exception handling for struct.error, an exception that DBFPY raises when trying to unpack a TSV table. Since it’s not a DBF file, we know it’s not a DBF and return False. Fixing an issue with the DBF set exporting test The DBF set export test needed a bit enabled (probably the writeable bit?) before the test would match the regression output. Updating dbf interface Updating the int/float class/type checking in the dbf format file. This allows for python2 and python3 compatibility. Tweaking dbfpy to work with python3 Altering a couple of imports. Updating dbf tests for binary data compatibility Making regression strings binary and improving debug messages for dbf assertion errors. Improving file handling for python 2 and 3 Updating DBF file handling for both python 2 and 3 in the _dbf interface. Adding a (seemingly) functional dbfpy for python3 I’ve made dbfpy python3 compatible! Tests appear to pass. A significant change was made to the format detection test whereby I made the input string a binary (bytes) string. If the string is not a bytes string by the time we try to detect the format, we try to decode the string as utf-8 (which admittedly might not be the safest thing to do) and try to decode anyways. Updating imports for tablib dbf interface Now importing python2 or python3 versions as appropriate. Updating dbf package references in compat.py Cleaning up debugging print statements Updating stream handling in dbf interface Factoring the open() call out of the py3 conditional and removing the temp file before returning the stream value. Adding dbfpy3 init.py I had apparently missed the dbfpy3 init file when committing dbfpy3. Adding dbfpy and dbfpy3 to setup.py's package list Switching test order of formats Putting dbf format testing ahead of TSV. In some of my tests with numeric DBF files, I encountered an issue where the ASCII horizontal tab character (0x09) would appear in a numeric DBF. Because of the order of tabular format imports, though, format detection would recognize it as a TSV and not as a DBF. Adding my name to AUTHORS. Adding a DBF property to tab lib core Documentation includes examples on how to explicitly load a DBF straight from a file and how to load a DBF from a binary string. Also, how to write the binary data to a file. Adding DBF format notes to README Adding exclamation point to DBF section title Matching formatting of XLS section Updating setup.py to match current dev state Setup.py had been updated since I forked the tablib repo, so I’m updating setup.py to match its current structure while still maintaining DBF compatibility. Fixed callable collumn test the test was sending a list instead of a function CORE CONTRIBUTORS 🍰 @iurisilvio v0.10.0 WHEELS 3.3, 3.4 makefile for WHEELS v0.10.0 history ALL Separate py2 and py3 packages to avoid installation errors. Fix #151 Running travis and tox with python 3.4.
183 lines
5.1 KiB
Python
183 lines
5.1 KiB
Python
#!/usr/bin/python
|
|
""".DBF creation helpers.
|
|
|
|
Note: this is a legacy interface. New code should use Dbf class
|
|
for table creation (see examples in dbf.py)
|
|
|
|
TODO:
|
|
- handle Memo fields.
|
|
- check length of the fields accoring to the
|
|
`http://www.clicketyclick.dk/databases/xbase/format/data_types.html`
|
|
|
|
"""
|
|
"""History (most recent first)
|
|
04-jul-2006 [als] added export declaration;
|
|
updated for dbfpy 2.0
|
|
15-dec-2005 [yc] define dbf_new.__slots__
|
|
14-dec-2005 [yc] added vim modeline; retab'd; added doc-strings;
|
|
dbf_new now is a new class (inherited from object)
|
|
??-jun-2000 [--] added by Hans Fiby
|
|
"""
|
|
|
|
__version__ = "$Revision: 1.4 $"[11:-2]
|
|
__date__ = "$Date: 2006/07/04 08:18:18 $"[7:-2]
|
|
|
|
__all__ = ["dbf_new"]
|
|
|
|
from .dbf import *
|
|
from .fields import *
|
|
from .header import *
|
|
from .record import *
|
|
|
|
class _FieldDefinition(object):
|
|
"""Field definition.
|
|
|
|
This is a simple structure, which contains ``name``, ``type``,
|
|
``len``, ``dec`` and ``cls`` fields.
|
|
|
|
Objects also implement get/setitem magic functions, so fields
|
|
could be accessed via sequence iterface, where 'name' has
|
|
index 0, 'type' index 1, 'len' index 2, 'dec' index 3 and
|
|
'cls' could be located at index 4.
|
|
|
|
"""
|
|
|
|
__slots__ = "name", "type", "len", "dec", "cls"
|
|
|
|
# WARNING: be attentive - dictionaries are mutable!
|
|
FLD_TYPES = {
|
|
# type: (cls, len)
|
|
"C": (DbfCharacterFieldDef, None),
|
|
"N": (DbfNumericFieldDef, None),
|
|
"L": (DbfLogicalFieldDef, 1),
|
|
# FIXME: support memos
|
|
# "M": (DbfMemoFieldDef),
|
|
"D": (DbfDateFieldDef, 8),
|
|
# FIXME: I'm not sure length should be 14 characters!
|
|
# but temporary I use it, cuz date is 8 characters
|
|
# and time 6 (hhmmss)
|
|
"T": (DbfDateTimeFieldDef, 14),
|
|
}
|
|
|
|
def __init__(self, name, type, len=None, dec=0):
|
|
_cls, _len = self.FLD_TYPES[type]
|
|
if _len is None:
|
|
if len is None:
|
|
raise ValueError("Field length must be defined")
|
|
_len = len
|
|
self.name = name
|
|
self.type = type
|
|
self.len = _len
|
|
self.dec = dec
|
|
self.cls = _cls
|
|
|
|
def getDbfField(self):
|
|
"Return `DbfFieldDef` instance from the current definition."
|
|
return self.cls(self.name, self.len, self.dec)
|
|
|
|
def appendToHeader(self, dbfh):
|
|
"""Create a `DbfFieldDef` instance and append it to the dbf header.
|
|
|
|
Arguments:
|
|
dbfh: `DbfHeader` instance.
|
|
|
|
"""
|
|
_dbff = self.getDbfField()
|
|
dbfh.addField(_dbff)
|
|
|
|
|
|
class dbf_new(object):
|
|
"""New .DBF creation helper.
|
|
|
|
Example Usage:
|
|
|
|
dbfn = dbf_new()
|
|
dbfn.add_field("name",'C',80)
|
|
dbfn.add_field("price",'N',10,2)
|
|
dbfn.add_field("date",'D',8)
|
|
dbfn.write("tst.dbf")
|
|
|
|
Note:
|
|
This module cannot handle Memo-fields,
|
|
they are special.
|
|
|
|
"""
|
|
|
|
__slots__ = ("fields",)
|
|
|
|
FieldDefinitionClass = _FieldDefinition
|
|
|
|
def __init__(self):
|
|
self.fields = []
|
|
|
|
def add_field(self, name, typ, len, dec=0):
|
|
"""Add field definition.
|
|
|
|
Arguments:
|
|
name:
|
|
field name (str object). field name must not
|
|
contain ASCII NULs and it's length shouldn't
|
|
exceed 10 characters.
|
|
typ:
|
|
type of the field. this must be a single character
|
|
from the "CNLMDT" set meaning character, numeric,
|
|
logical, memo, date and date/time respectively.
|
|
len:
|
|
length of the field. this argument is used only for
|
|
the character and numeric fields. all other fields
|
|
have fixed length.
|
|
FIXME: use None as a default for this argument?
|
|
dec:
|
|
decimal precision. used only for the numric fields.
|
|
|
|
"""
|
|
self.fields.append(self.FieldDefinitionClass(name, typ, len, dec))
|
|
|
|
def write(self, filename):
|
|
"""Create empty .DBF file using current structure."""
|
|
_dbfh = DbfHeader()
|
|
_dbfh.setCurrentDate()
|
|
for _fldDef in self.fields:
|
|
_fldDef.appendToHeader(_dbfh)
|
|
|
|
_dbfStream = open(filename, "wb")
|
|
_dbfh.write(_dbfStream)
|
|
_dbfStream.close()
|
|
|
|
|
|
if (__name__=='__main__'):
|
|
# create a new DBF-File
|
|
dbfn=dbf_new()
|
|
dbfn.add_field("name",'C',80)
|
|
dbfn.add_field("price",'N',10,2)
|
|
dbfn.add_field("date",'D',8)
|
|
dbfn.write("tst.dbf")
|
|
# test new dbf
|
|
print("*** created tst.dbf: ***")
|
|
dbft = Dbf('tst.dbf', readOnly=0)
|
|
print(repr(dbft))
|
|
# add a record
|
|
rec=DbfRecord(dbft)
|
|
rec['name']='something'
|
|
rec['price']=10.5
|
|
rec['date']=(2000,1,12)
|
|
rec.store()
|
|
# add another record
|
|
rec=DbfRecord(dbft)
|
|
rec['name']='foo and bar'
|
|
rec['price']=12234
|
|
rec['date']=(1992,7,15)
|
|
rec.store()
|
|
|
|
# show the records
|
|
print("*** inserted 2 records into tst.dbf: ***")
|
|
print(repr(dbft))
|
|
for i1 in range(len(dbft)):
|
|
rec = dbft[i1]
|
|
for fldName in dbft.fieldNames:
|
|
print('%s:\t %s'%(fldName, rec[fldName]))
|
|
print()
|
|
dbft.close()
|
|
|
|
# vim: set et sts=4 sw=4 :
|