Thursday 14 June 2012

Python: Breaking apart large files into smaller chunks

Something that I find really neat with Python is its flexibility and strong support in the online community.  I found a really good example somewhere on how to do this and I adapted it to my own needs...  like most programmers ;)
import os
def UnpileFile(src, outPATH, parts):
    if not os.path.isdir(outPATH): os.mkdir(outPATH)
    f = open(src, 'rb')
    data = f.read()
    f.close()
    byts = len(data)
    inc = (byts+4)/int(parts)
    filenames = []
    for i in range(0, byts+1, inc):
        fn1 = outPATH + "\\file %s" % i
        filenames.append(fn1)
        f = open(fn1, 'wb')
        f.write(data[i:i+inc])
        f.close()
def CompileFile(srcPATH, outFILE):
    if not os.path.isdir(srcPATH): os.mkdir(srcPATH)
    dataList = []
    for root, dirs, files in os.walk(srcPATH):
        for fn in files:
            f = open(str(root+'\\'+fn), 'rb')
            dataList.append(f.read())
            f.close()
    f = open(outFILE, 'wb')
    for data in dataList:
        f.write(data)
    f.close()
if __name__ == '__main__':
    pass
There's not much to import which is a plus... ;)

So say you had an ISO image... you only have 2 2gb usb drives?  no problem... break these guys up into 2 parts by saying;
src = the path to the ISO you want to break up
outPATH = the directory you want to save the broken pieces to
parts = the number of parts you want the files split into
UnpileFile(src, outPATH, parts)


















And to put humpty dumpty back together again... so to speak... you just say:

srcPATH = the directory that has the broken files
outFILE = the name you want to give your patched up ISO...
CompileFile(srcPATH, outFILE)
*NOTE... I am pretty sure I specified mkdir and not makedirs, there is a difference...

http://docs.python.org/library/os.html?highlight=os.mkdir#os.mkdir

No comments:

Post a Comment