This is a Library written in (unmanaged) C/C++ that can be used in your .NET (C# example available) applications to decompress and compress the compressed data from the savegame.
Make sure to give credits if used in your application.
Update:
So, lzopro gave me a hard time as it's not ment to be used the way i wanted it. Took some time to fix this but here is a new version that should work as expected.
// result now contains the real decompressed size and decompressed_buffer contains the decompressed data.
}
Available compression methods:
Code:
enum {
// decompressors
COMP_NONE = 0, // No compression
COMP_ZLIB, // RFC 1950
COMP_DEFLATE, // RFC 1951
COMP_LZO1, // LZO 1 Freeware
COMP_LZO1A, // LZO 1a Freeware
COMP_LZO1B, // LZO 1b (safe with overrun) Freeware
COMP_LZO1C, // LZO 1c (safe with overrun) Freeware
COMP_LZO1F, // LZO 1f (safe with overrun) Freeware
COMP_LZO1X, // LZO 1x (safe with overrun) Freeware
COMP_LZO1Y, // LZO 1y (safe with overrun) Freeware
COMP_LZO1Z, // LZO 1z (safe with overrun) Freeware
COMP_LZO2A, // LZO 2a (safe with overrun) Freeware
COMP_LZOPRO1X, // LZOPRO 1x (safe with overrun) Freeware
COMP_LZOPRO1Y, // LZOPRO 1y (safe with overrun) Freeware
// compressors
COMP_LZO1_COMPRESS, // LZO 1 Freeware
COMP_LZO1_99_COMPRESS, // better compression ratio at the cost of more memory and time
COMP_LZO1A_COMPRESS, // LZO 1a Freeware
COMP_LZO1A_99_COMPRESS, // better compression ratio at the cost of more memory and time
COMP_LZO1B_COMPRESS, // LZO 1b Freeware (Valid compression level: 1..9)
COMP_LZO1B_99_COMPRESS, // better compression ratio at the cost of more memory and time
COMP_LZO1B_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO1C_COMPRESS, // LZO 1c Freeware (Valid compression level: 1..9)
COMP_LZO1C_99_COMPRESS, // better compression ratio at the cost of more memory and time
COMP_LZO1C_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO1F_COMPRESS, // LZO 1f Freeware
COMP_LZO1F_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO1X_COMPRESS, // LZO 1x Freeware
COMP_LZO1X_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO1Y_COMPRESS, // LZO 1y Freeware
COMP_LZO1Y_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO1Z_COMPRESS, // LZO 1z Freeware
COMP_LZO1Z_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO2A_COMPRESS, // LZO 2a Freeware
COMP_LZO2A_999_COMPRESS, // even better compression ratio at the cost of more memory and time
COMP_LZOPRO1X_COMPRESS, // LZOPRO 1x (Valid compression level: 1..10)
COMP_LZOPRO1Y_COMPRESS, // LZOPRO 1y (Valid compression level: 1..10)
};
09-16-2012
CloudStrife7x
Re: Generic Xbox360 Compression Library
awesome work on this fair :076::029:
09-16-2012
thegutta
Excellent work Fairchild :)
09-17-2012
Serious_M
Re: Generic Xbox360 Compression Library
very nice work as always fairchild
09-19-2012
kill_seth
Re: Generic Xbox360 Compression Library
Can't seem to get it working in VB.Net
Code:
<DllImport("compress.dll", CallingConvention:=CallingConvention.Cdecl)> _
Public Shared Function lzo_decompress(<Out> compressed_buffer As Byte(), compressed_size As UInteger, <[In]> decompressed_buffer As Byte(), decompressed_size As UInteger, type As UInteger) As UInteger
End Function
Private Function Decompress(compressedData As Byte()) As Byte()
Dim decompressedSize As UInt32 = dSize
Dim decompressedData As Byte() = New Byte(decompressedSize) {}
Dim compressedSize As UInt32 = cSize
lzo_decompress(compressedData, compressedSize, decompressedData, decompressedSize, 0)
Return decompressedData
End Function
When I go to call the function I get: Unable to load DLL 'compress.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
I placed the dll in the Application.StartUpPath and in Windows\System32
09-19-2012
PUR3 RAF3X
Re: Generic Xbox360 Compression Library
Yes the same prob here. I have try use another ways but nothing work with this dll.
Code:
Imports PackageIO
Imports System.Text
Imports System
Imports System.Runtime.InteropServices
Code:
Public Declare Function lzo_decompress Lib "C:\Users\****\Documents\Visual Studio 2010\Projects\WindowsApplication5\WindowsApplication5\obj\x86\Debug\compress.dll" (ByVal Compressed_Buffer() As Byte, ByVal Compressed_Buffersize As UInteger, ByVal decompressed_buffer() As Byte, ByVal decompressed_size As UInteger, ByVal type As UInteger) As Integer
Code:
<DllImport("compress.dll", CharSet:=CharSet.Auto, CallingConvention:=CallingConvention.Cdecl)> _
Public Shared Function lzo_decompress(<Out()> compressed_buffer As Byte(), compressed_size As UInteger, <[In]()> decompressed_buffer As Byte(), decompressed_size As UInteger, type As UInteger) As UInteger
End Function
Code:
Dim FilePath As String
Dim Compressed_Buffer() As Byte
Dim decompressed_size As UInteger
Dim compressed_size As UInteger
Private Sub Decompressdata()
Dim reader As New PackageIO.Reader(FilePath, Endian.Big)
reader.Position = 28
decompressed_size = reader.ReadInt32()
Dim decompressed_buffer As Byte() = New Byte((decompressed_size)) {}
Compressed_Buffer = reader.ReadBytes(reader.Length, Endian.Little)
compressed_size = Compressed_Buffer.Length
decompressed_buffer = {(lzo_decompress(Compressed_Buffer, compressed_size, decompressed_buffer, decompressed_size, 0))}
System.IO.File.WriteAllBytes(Application.StartupPath + "/SaveGame.dat", decompressed_buffer)
End Sub
09-19-2012
Jappi88
Re: Generic Xbox360 Compression Library
U guys can try whatever u want...it wont work...its not the way u coded it...fairchild forget to inlcude the maindll.
09-21-2012
PUR3 RAF3X
Re: Generic Xbox360 Compression Library
Whats happen with this dll?
09-21-2012
Jappi88
Re: Generic Xbox360 Compression Library
Quote:
Originally Posted by PUR3 RAF3X
Whats happen with this dll?
Be Patient...Fairchild will fix it when he gets the time.
Also dont get your hopes up , u most probebly wont be able to desompress the save using the compress.dll(LzoPro)
09-26-2012
fairchild
Re: Generic Xbox360 Compression Library
Compression library has been updated now, hope everything works smooth now.
Note the new argument (uint compress_level) on the compress function.
Valid compress levels are (LZO Freeware: 1-9, LZO PRO: 1-10)
09-26-2012
mojobojo
Re: Generic Xbox360 Compression Library
The readme has a typo, the VB code example is copied twice. Also wouldn't we need an h file for the lib file to include the lib file? Anways, great job.
09-26-2012
kill_seth
Re: Generic Xbox360 Compression Library
Just tested out the compression, and it seems either I'm doing something wrong or it's still not working properly.
Code:
<DllImport("compress.dll", CallingConvention:=CallingConvention.Cdecl)> _
Public Shared Function lzo_compress(<Out> decompressed_buffer As Byte(), decompressed_size As UInteger, <[In]> compressed_buffer As Byte(), compressed_size As UInteger, type As UInteger, compression_level As UInteger) As UInteger
End Function
Private Function Compress(decompressedData As Byte()) As Byte()
Dim compressedData As Byte() = New Byte(compressedSize - 1) {}
lzo_compress(decompressedData, decompressedSize, compressedData, compressedSize, 12, 1)
Return compressedData
End Function
It's just returning a null byte array. I tried doing different compression levels, and also tried 13 but got the same result.
09-26-2012
fairchild
Re: Generic Xbox360 Compression Library
The library seems to be unstable, need to look into more error handling when sending signals from unmanaged code to managed, but the library works in my example C# tool i made really fast. Perhaps the sourcecode below helps:
Code:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using System.IO;
namespace example
{
public partial class Form1 : Form
{
enum CompressionMethods {
// decompressors
COMP_NONE = 0, // No compression
COMP_ZLIB, // RFC 1950
COMP_DEFLATE, // RFC 1951
COMP_LZO1, // LZO 1 Freeware
COMP_LZO1A, // LZO 1a Freeware
COMP_LZO1B, // LZO 1b (safe with overrun) Freeware
COMP_LZO1C, // LZO 1c (safe with overrun) Freeware
COMP_LZO1F, // LZO 1f (safe with overrun) Freeware
COMP_LZO1X, // LZO 1x (safe with overrun) Freeware
COMP_LZO1Y, // LZO 1y (safe with overrun) Freeware
COMP_LZO1Z, // LZO 1z (safe with overrun) Freeware
COMP_LZO2A, // LZO 2a (safe with overrun) Freeware
COMP_LZOPRO1X, // LZOPRO 1x (safe with overrun) Freeware
COMP_LZOPRO1Y, // LZOPRO 1y (safe with overrun) Freeware
// compressors
COMP_LZO1_COMPRESS, // LZO 1 Freeware
COMP_LZO1_99_COMPRESS, // better compression ratio at the cost of more memory and time
COMP_LZO1A_COMPRESS, // LZO 1a Freeware
COMP_LZO1A_99_COMPRESS, // better compression ratio at the cost of more memory and time
COMP_LZO1B_COMPRESS, // LZO 1b Freeware (Valid compression level: 1..9)
COMP_LZO1B_99_COMPRESS, // better compression ratio at the cost of more memory and time
COMP_LZO1B_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO1C_COMPRESS, // LZO 1c Freeware (Valid compression level: 1..9)
COMP_LZO1C_99_COMPRESS, // better compression ratio at the cost of more memory and time
COMP_LZO1C_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO1F_COMPRESS, // LZO 1f Freeware
COMP_LZO1F_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO1X_COMPRESS, // LZO 1x Freeware
COMP_LZO1X_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO1Y_COMPRESS, // LZO 1y Freeware
COMP_LZO1Y_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO1Z_COMPRESS, // LZO 1z Freeware
COMP_LZO1Z_999_COMPRESS,// even better compression ratio at the cost of more memory and time
COMP_LZO2A_COMPRESS, // LZO 2a Freeware
COMP_LZO2A_999_COMPRESS, // even better compression ratio at the cost of more memory and time
COMP_LZOPRO1X_COMPRESS, // LZOPRO 1x (Valid compression level: 1..10)
COMP_LZOPRO1Y_COMPRESS, // LZOPRO 1y (Valid compression level: 1..10)
}
public byte[] ReadAllBytes(string fileName, int offset)
{
byte[] buffer = null;
using (FileStream fs = new FileStream(fileName, FileMode.Open, FileAccess.Read))
{
//Console.Out.WriteLine("fs.Length: " + fs.Length + "\n");
//Console.Out.WriteLine("offset: " + offset + "\n");
buffer = new byte[(fs.Length - offset)];
fs.Seek(offset, SeekOrigin.Begin);
fs.Read(buffer, 0, (int)(fs.Length - offset));
}
return buffer;
}
richTextBox1.Text += "\nCompress result: " + result + "\n";
for (int i = 0; i < 128; i++)
richTextBox1.Text += compressed_buffer[i].ToString("X2");
/*
for (int i = 0; i < result; i++)
richTextBox1.Text += decompressed_buffer[i].ToString("X2");
*/
}
}
}
09-26-2012
PUR3 RAF3X
Re: Generic Xbox360 Compression Library
I try again thanks hope we can De/Compress Borderland 2 ;)
09-26-2012
PUR3 RAF3X
Re: Generic Xbox360 Compression Library
No idea why but the decompressedData return allways no data? Any idea?
09-26-2012
fairchild
Re: Generic Xbox360 Compression Library
Quote:
Originally Posted by PUR3 RAF3X
No idea why but the decompressedData return allways no data? Any idea?
Don't know, perhaps it's correct but why [] on the In and not the Out and should the [] be there for VB?
Code:
<Out> decompressed_buffer As Byte(), decompressed_size As UInteger, <[In]> compressed_buffer
09-26-2012
Jappi88
Re: Generic Xbox360 Compression Library
this code is for vb to call the unmanaged decompress function >>:
<DllImport("compress.dll", CallingConvention:=CallingConvention.Cdecl)> _
Public Shared Function lzo_compress(<Out()> ByVal decompressed_buffer As Byte(), ByVal decompressed_size As Integer, <[In]()> ByVal compressed_buffer As Byte(), ByVal compressed_size As Integer, ByVal type As Integer) As Integer
End Function
if the decompress Returns nulled or invalid data means the file ur decompressing does not use that type.
so to check if ur decompressing whent like it suposed to be, could be done like this :
'declare a varaible as integer to use for return result from the decompressor
Dim Result as Integer = lzo_decompress(buffer, buffer.Length, Decompressed, Length, 8)
if Result > -1 then msgbox("Succesfull Decompression") else msgbox("Failed decompressing file!")
if the decompressing when succesfully it will return the decompressed length else it will return a negative value. (-1)
09-26-2012
PUR3 RAF3X
Re: Generic Xbox360 Compression Library
lmao my vault I have do a noob step this is why I have trouble but guys it's work and I can Decompress & Compress the Borderland 2 save. When kill_seph not update his Tool then I release my one and big big thanks to fairchild. All credits to him ;)
Namespace example
Public Partial Class Form1
Inherits Form
Private Enum CompressionMethods
' decompressors
COMP_NONE = 0
' No compression
COMP_ZLIB
' RFC 1950
COMP_DEFLATE
' RFC 1951
COMP_LZO1
' LZO 1 Freeware
COMP_LZO1A
' LZO 1a Freeware
COMP_LZO1B
' LZO 1b (safe with overrun) Freeware
COMP_LZO1C
' LZO 1c (safe with overrun) Freeware
COMP_LZO1F
' LZO 1f (safe with overrun) Freeware
COMP_LZO1X
' LZO 1x (safe with overrun) Freeware
COMP_LZO1Y
' LZO 1y (safe with overrun) Freeware
COMP_LZO1Z
' LZO 1z (safe with overrun) Freeware
COMP_LZO2A
' LZO 2a (safe with overrun) Freeware
COMP_LZOPRO1X
' LZOPRO 1x (safe with overrun) Freeware
COMP_LZOPRO1Y
' LZOPRO 1y (safe with overrun) Freeware
' compressors
COMP_LZO1_COMPRESS
' LZO 1 Freeware
COMP_LZO1_99_COMPRESS
' better compression ratio at the cost of more memory and time
COMP_LZO1A_COMPRESS
' LZO 1a Freeware
COMP_LZO1A_99_COMPRESS
' better compression ratio at the cost of more memory and time
COMP_LZO1B_COMPRESS
' LZO 1b Freeware (Valid compression level: 1..9)
COMP_LZO1B_99_COMPRESS
' better compression ratio at the cost of more memory and time
COMP_LZO1B_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO1C_COMPRESS
' LZO 1c Freeware (Valid compression level: 1..9)
COMP_LZO1C_99_COMPRESS
' better compression ratio at the cost of more memory and time
COMP_LZO1C_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO1F_COMPRESS
' LZO 1f Freeware
COMP_LZO1F_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO1X_COMPRESS
' LZO 1x Freeware
COMP_LZO1X_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO1Y_COMPRESS
' LZO 1y Freeware
COMP_LZO1Y_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO1Z_COMPRESS
' LZO 1z Freeware
COMP_LZO1Z_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO2A_COMPRESS
' LZO 2a Freeware
COMP_LZO2A_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZOPRO1X_COMPRESS
' LZOPRO 1x (Valid compression level: 1..10)
COMP_LZOPRO1Y_COMPRESS
' LZOPRO 1y (Valid compression level: 1..10)
End Enum
Public Function ReadAllBytes(fileName As String, offset As Integer) As Byte()
Dim buffer As Byte() = Nothing
Using fs As New FileStream(fileName, FileMode.Open, FileAccess.Read)
'Console.Out.WriteLine("fs.Length: " + fs.Length + "\n");
'Console.Out.WriteLine("offset: " + offset + "\n");
buffer = New Byte((fs.Length - offset) - 1) {}
fs.Seek(offset, SeekOrigin.Begin)
fs.Read(buffer, 0, CInt(fs.Length - offset))
End Using
Return buffer
End Function
Public Sub New()
InitializeComponent()
End Sub
<DllImport("compress.dll", CallingConvention := CallingConvention.Cdecl)> _
Public Shared Function lzo_decompress(<Out> compressed_buffer As Byte(), compressed_size As UInteger, <[In]> decompressed_buffer As Byte(), decompressed_size As UInteger, method As UInteger) As UInteger
End Function
<DllImport("compress.dll", CallingConvention := CallingConvention.Cdecl)> _
Public Shared Function lzo_compress(<Out> decompressed_buffer As Byte(), decompressed_size As UInteger, <[In]> compressed_buffer As Byte(), compressed_size As UInteger, method As UInteger, compress_level As UInteger) As UInteger
End Function
Private Sub button1_Click(sender As Object, e As EventArgs)
Dim Pfad As String = String.Empty
Dim openFileDialog1 As New OpenFileDialog()
openFileDialog1.Filter = "Save (*.sav)|*.sav|All files (*.*)|*.*"
If openFileDialog1.ShowDialog() = DialogResult.OK Then
Pfad = openFileDialog1.FileName
End If
Dim decompressed_size As UInt32 = 24875
' Static for now on my testfile
Dim decompressed_buffer As Byte() = New Byte(decompressed_size - 1) {}
Dim compressed_buffer As Byte() = ReadAllBytes(Pfad, &H18)
Dim compressed_size As UInteger = CUInt(compressed_buffer.Length)
Dim result As UInteger = lzo_decompress(compressed_buffer, compressed_size, decompressed_buffer, decompressed_size, CUInt(CompressionMethods.COMP_LZOPRO1X))
richTextBox1.Text += vbLf & "Decompress result: " & result & vbLf
For i As Integer = 0 To 127
richTextBox1.Text += decompressed_buffer(i).ToString("X2")
Next
decompressed_size = result
result = lzo_compress(decompressed_buffer, decompressed_size, compressed_buffer, compressed_size, CUInt(CompressionMethods.COMP_LZOPRO1X_COMPRESS), 10)
richTextBox1.Text += vbLf & "Compress result: " & result & vbLf
For i As Integer = 0 To 127
richTextBox1.Text += compressed_buffer(i).ToString("X2")
Next
'
' for (int i = 0; i < result; i++)
' richTextBox1.Text += decompressed_buffer[i].ToString("X2");
'
End Sub
End Class
End Namespace
09-26-2012
kill_seth
Re: Generic Xbox360 Compression Library
Did some more testing, and it seems the library isn't working correctly with the compression.
Code:
<DllImport("compress.dll", CallingConvention:=CallingConvention.Cdecl)> _
Public Shared Function lzo_compress(<Out> decompressed_buffer As Byte(), decompressed_size As UInteger, <[In]> compressed_buffer As Byte(), compressed_size As UInteger, type As UInteger, compression_level As UInteger) As UInteger
End Function
If OFD.ShowDialog = Windows.Forms.DialogResult.OK Then
gReader = New GPDPackage.GPD(OFD.FileName)
iReader = New IOPackage.RWStream(gReader.ExtractSaveData(GPDPackage.GPD.SaveData.TitleSpecific1))
SHA1 = iReader.ReadBytes(&H14)
decompressedSize = iReader.ReadUInt32()
Dim decompressedData As Byte() = New Byte(decompressedSize - 1) {}
Dim compressedData As Byte() = iReader.ReadBytes(iReader.Length - iReader.Position)
compressedSize = compressedData.Length
File.WriteAllBytes("Profile(compressed)", compressedData)
Dim result = lzo_decompress(compressedData, compressedSize, decompressedData, decompressedSize, 12)
Dim result2 = lzo_compress(decompressedData, decompressedSize, compressedData, compressedSize, 12, 10)
File.WriteAllBytes("Profile(decompressed)", decompressedData)
File.WriteAllBytes("Profile(recompressed)", compressedData)
End If
Doing that both the Profile(compressed) and Profile(recompressed) files are exactly the same. So at first, I thought I finally got it to work and decided to try something else. I edited the decompressed data and wrote a 1 to the first uint32 entry, then tried to recompress it. Turns out, both the files were exactly the same again. I even tried creating a recompressedData and recompressedSize, but then I just ended up with a null bytes once again. Thanks for all your hard work, and I hope we can figure this out!
09-26-2012
PUR3 RAF3X
Re: Generic Xbox360 Compression Library
Yes this is hard and make me sick.
09-27-2012
PUR3 RAF3X
Re: Generic Xbox360 Compression Library
@kill the vault why it's return 0 is when you set CompressionMethods to 0 or none then it's return all bytes with "0" use for decompress CompressionMethods.COMP_LZOPRO1X or try around with another Enums. But I have the best result with Decompress this data using my own LZO 2.06 with custom setting.
09-27-2012
eyebiz
Re: Generic Xbox360 Compression Library
I must be missing something. How could you possibly know the size "decompressed_size" of the decompressed_buffer before decompression?
Edit: When working with a pre-compressed file, aka the BL2 save.
09-27-2012
Jappi88
Re: Generic Xbox360 Compression Library
Quote:
Originally Posted by eyebiz
I must be missing something. How could you possibly know the size "decompressed_size" of the decompressed_buffer before decompression?
Because in some compressions the decompressed size is discripted inside the data.
Lzo works that way.
@Pur3
there is nothing wrong with compress.dll(lzopro) works like it should... It jyst doesnt have the suported compression type for borderlands 2.
09-28-2012
PUR3 RAF3X
Re: Generic Xbox360 Compression Library
Quote:
Originally Posted by Jappi88
Because in some compressions the decompressed size is discripted inside the data.
Lzo works that way.
@Pur3
there is nothing wrong with compress.dll(lzopro) works like it should... It jyst doesnt have the suported compression type for borderlands 2.
Ahh ok. Yes I see this the dll do the job and I have include this to my Tool so the guys can use it and try self around. The Tool support Packzip/offzip and this lib ;) work in progress are xextool I now many will love this.
09-28-2012
PUR3 RAF3X
Re: Generic Xbox360 Compression Library
Quote:
Originally Posted by eyebiz
I must be missing something. How could you possibly know the size "decompressed_size" of the decompressed_buffer before decompression?
Edit: When working with a pre-compressed file, aka the BL2 save.
Quote:
LZO implements a number of algorithms with the following features:
Decompression is simple and *very* fast.
Requires no memory for decompression.
Compression is pretty fast.
Requires 64 kB of memory for compression.
Allows you to dial up extra compression at a speed cost in the compressor. The speed of the decompressor is not reduced.
Includes compression levels for generating pre-compressed data which achieve a quite competitive compression ratio.
There is also a compression level which needs only 8 kB for compression.
Algorithm is thread safe.
Algorithm is lossless.
LZO supports overlapping compression and in-place decompression.
Maybe this help you.
09-30-2012
PUR3 RAF3X
Re: Generic Xbox360 Compression Library
For Borderlands you can do the Decompress like this sample: But have any try compress Decompress big files over 100mb? This is my first time I work with Compress so any help welcome. I have try fill the buffer with 8kb and copy the compressed 6kb array to the 8kb buffer and try decompress this. The result are the same 6kb or 8kb lzo skip all "0" after the 6kb and return the same decompressed bytes.
EDIT:I have my answer and I work on my own Module doing this with files over 8GB ;) Using custom header for files over 512MB. You guys can downloade the source later.
Code:
Private Sub DecompressButton_Click_1(sender As System.Object, e As System.EventArgs) Handles DecompressButton.Click
try
If Not DecompressInputPath = Nothing Then
Writer = New BinaryWriter(New FileStream(Me.DecompressSavePath, FileMode.Create, FileAccess.Write, FileShare.Write))
Dim mblock As Integer = 8192
Dim bsize As Integer = 8192 * 8 '/64kb 8kb * 8 = 64kb
Dim buffer As Byte() = New Byte(bsize - 1) {}
Dim result As UInteger = 0
Dim compressed_buffer As Byte() = ReadBufferBytes(DecompressInputPath, 24,mblock)
result = lzo_decompress(compressed_buffer, compressed_buffer.Length, buffer, buffer.Length, _
CUInt(CompressionMethodsdecompress(ComboBox1.Text, True)))
If Not result = UInteger.MaxValue Then
Writer.Seek(&H0, SeekOrigin.Begin)
Writer.Write(buffer, 0, CInt(result))
Else
MsgBox("Decompress Failed")
End If
End If
Writer.Flush()
Writer.Close()
Catch ex As Exception
MsgBox(ex.Message)
End Try
End Sub
10-07-2012
JizzaBeez
Re: Generic Xbox360 Compression Library
Farichild: how do I know the "compressed_size" when using lzo_compress ? Wouldn't the size be determined by input, method and level?
10-08-2012
eyebiz
Re: Generic Xbox360 Compression Library
Quote:
Originally Posted by JizzaBeez
Farichild: how do I know the "compressed_size" when using lzo_compress ? Wouldn't the size be determined by input, method and level?
I used the same as "decompressed_size" and then just resized the array to the "result" value after compression.
10-08-2012
PUR3 RAF3X
Re: Generic Xbox360 Compression Library
Yes the compress methode have different output size. I have no idea if there any another way doing this but my way I doing this is create a buffer with bytes and use UINT result for the info "Compress size". Thats all you can do the same again with the right info from result or you copy the compress data array from the buffer you have created for the info or write only the size from byte array there more then 1 way doing this and Im sure you found your right way what you need ;)
Im not the dude hide all my source so here my "WIP" Module for Compress big files with custom header this store each compress buffer size how many blocks and the memblock size. Only when the file size more then 256Mb.
I have only probs with the damn Progress inside the R_textbox for compress the data. reade byte work well.
<DllImport("Plugins\compress.dll", CallingConvention:=CallingConvention.Cdecl)> _
Public Function lzo_decompress(<Out()> compressed_buffer As Byte(), compressed_size As UInteger, <[In]()> decompressed_buffer As Byte(), decompressed_size As UInteger, method As UInteger) As UInteger
End Function
<DllImport("Plugins\compress.dll", CallingConvention:=CallingConvention.Cdecl)> _
Public Function lzo_compress(<Out()> decompressed_buffer As Byte(), decompressed_size As UInteger, <[In]()> compressed_buffer As Byte(), compressed_size As UInteger, method As UInteger, compress_level As UInteger) As UInteger
End Function
Private bw As BinaryWriter
Private strb As New System.Text.StringBuilder
Private multiblocks As Boolean = False
Dim InputPath As String
Dim OutputPath As String
Dim endfile As Boolean = False
Dim Mem As Integer = 268435456
Dim calculatedblocks As Integer
Dim Lastblocksize As UInteger = 0
Dim Headersize As Integer = 0
Dim blockstartpos As String = &H0
Private Enum CompressionMethods
' decompressors
COMP_NONE = 0
' No compression
COMP_ZLIB
' RFC 1950
COMP_DEFLATE
' RFC 1951
COMP_LZO1
' LZO 1 Freeware
COMP_LZO1A
' LZO 1a Freeware
COMP_LZO1B
' LZO 1b (safe with overrun) Freeware
COMP_LZO1C
' LZO 1c (safe with overrun) Freeware
COMP_LZO1F
' LZO 1f (safe with overrun) Freeware
COMP_LZO1X
' LZO 1x (safe with overrun) Freeware
COMP_LZO1Y
' LZO 1y (safe with overrun) Freeware
COMP_LZO1Z
' LZO 1z (safe with overrun) Freeware
COMP_LZO2A
' LZO 2a (safe with overrun) Freeware
COMP_LZOPRO1X
' LZOPRO 1x (safe with overrun) Freeware
COMP_LZOPRO1Y
' LZOPRO 1y (safe with overrun) Freeware
' compressors
COMP_LZO1_COMPRESS
' LZO 1 Freeware
COMP_LZO1_99_COMPRESS
' better compression ratio at the cost of more memory and time
COMP_LZO1A_COMPRESS
' LZO 1a Freeware
COMP_LZO1A_99_COMPRESS
' better compression ratio at the cost of more memory and time
COMP_LZO1B_COMPRESS
' LZO 1b Freeware (Valid compression level: 1..9)
COMP_LZO1B_99_COMPRESS
' better compression ratio at the cost of more memory and time
COMP_LZO1B_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO1C_COMPRESS
' LZO 1c Freeware (Valid compression level: 1..9)
COMP_LZO1C_99_COMPRESS
' better compression ratio at the cost of more memory and time
COMP_LZO1C_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO1F_COMPRESS
' LZO 1f Freeware
COMP_LZO1F_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO1X_COMPRESS
' LZO 1x Freeware
COMP_LZO1X_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO1Y_COMPRESS
' LZO 1y Freeware
COMP_LZO1Y_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO1Z_COMPRESS
' LZO 1z Freeware
COMP_LZO1Z_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZO2A_COMPRESS
' LZO 2a Freeware
COMP_LZO2A_999_COMPRESS
' even better compression ratio at the cost of more memory and time
COMP_LZOPRO1X_COMPRESS
' LZOPRO 1x (Valid compression level: 1..10)
COMP_LZOPRO1Y_COMPRESS
' LZOPRO 1y (Valid compression level: 1..10)
End Enum
Sub Compressfile(CompressInputPath As String, CompressSavePath As String)
InputPath = CompressInputPath
OutputPath = CompressSavePath
Calculatefilecompression()
StartCompression()
'Verifyheader()
End Sub
#Region "Compress Methoden"
Private Function CompressionMethodsdecompress(p1 As Object, ByVal ReturnInt As Boolean) As Object
If ReturnInt Then
Select Case p1
Case "COMP_LZOPRO1X"
Return CompressionMethods.COMP_LZOPRO1X
Case "COMP_LZOPRO1Y"
Return CompressionMethods.COMP_LZOPRO1Y
Case "COMP_LZO2A"
Return CompressionMethods.COMP_LZO2A
Case "COMP_LZO1Z"
Return CompressionMethods.COMP_LZO1Z
Case "COMP_LZO1Y"
Return CompressionMethods.COMP_LZO1Y
Case "COMP_LZO1X"
Return CompressionMethods.COMP_LZO1X
Case "COMP_LZO1F"
Return CompressionMethods.COMP_LZO1F
Case "COMP_LZO1C"
Return CompressionMethods.COMP_LZO1C
Case "COMP_LZO1B"
Return CompressionMethods.COMP_LZO1B
Case "COMP_LZO1A"
Return CompressionMethods.COMP_LZO1A
Case "COMP_LZO1"
Return CompressionMethods.COMP_LZO1
Case "COMP_DEFLATE"
Return CompressionMethods.COMP_DEFLATE
Case "COMP_ZLIB"
Return CompressionMethods.COMP_ZLIB
Case "COMP_NONE"
Return CompressionMethods.COMP_NONE
Case Else
xb360Generic.RichTextBox1.AppendText("Wrong CompressionMethods" & xb360Generic.ComboBox1.Text.ToString)
Return False
End Select
End If
Return Nothing
End Function
Private Function CompressionMethodscompress(p1 As Object, ByVal ReturnInt As Boolean) As Object 'Compress
If ReturnInt Then
Select Case p1
Case "COMP_LZO1_COMPRESS"
Return CompressionMethods.COMP_LZO1_COMPRESS
Case "COMP_LZO1_99_COMPRESS"
Return CompressionMethods.COMP_LZO1_99_COMPRESS
Case "COMP_LZO1A_COMPRESS"
Return CompressionMethods.COMP_LZO1A_COMPRESS
Case "COMP_LZO1A_99_COMPRESS"
Return CompressionMethods.COMP_LZO1A_99_COMPRESS
Case "COMP_LZO1B_COMPRESS"
Return CompressionMethods.COMP_LZO1B_COMPRESS
Case "COMP_LZO1B_99_COMPRESS"
Return CompressionMethods.COMP_LZO1B_99_COMPRESS
Case "COMP_LZO1B_999_COMPRESS"
Return CompressionMethods.COMP_LZO1B_999_COMPRESS
Case "COMP_LZO1C_COMPRESS"
Return CompressionMethods.COMP_LZO1C_COMPRESS
Case "COMP_LZO1C_99_COMPRESS"
Return CompressionMethods.COMP_LZO1C_99_COMPRESS
Case "COMP_LZO1C_999_COMPRESS"
Return CompressionMethods.COMP_LZO1C_999_COMPRESS
Case "COMP_LZO1F_COMPRESS"
Return CompressionMethods.COMP_LZO1F_COMPRESS
Case "COMP_LZO1F_999_COMPRESS"
Return CompressionMethods.COMP_LZO1F_999_COMPRESS
Case "COMP_LZO1X_COMPRESS"
Return CompressionMethods.COMP_LZO1X_COMPRESS
Case "COMP_LZO1X_999_COMPRESS"
Return CompressionMethods.COMP_LZO1X_999_COMPRESS
Case "COMP_LZO1Y_COMPRESS"
Return CompressionMethods.COMP_LZO1Y_COMPRESS
Case "COMP_LZO1Y_999_COMPRESS"
Return CompressionMethods.COMP_LZO1Y_999_COMPRESS
Case "COMP_LZO1Z_COMPRESS"
Return CompressionMethods.COMP_LZO1Z_COMPRESS
Case "COMP_LZO1Z_999_COMPRESS"
Return CompressionMethods.COMP_LZO1Z_999_COMPRESS
Case "COMP_LZO2A_COMPRESS"
Return CompressionMethods.COMP_LZO2A_COMPRESS
Case "COMP_LZO2A_999_COMPRESS"
Return CompressionMethods.COMP_LZO2A_999_COMPRESS
Case "COMP_LZOPRO1X_COMPRESS"
Return CompressionMethods.COMP_LZOPRO1X_COMPRESS
Case "COMP_LZOPRO1Y_COMPRESS"
Return CompressionMethods.COMP_LZOPRO1Y_COMPRESS
Case Else
xb360Generic.RichTextBox1.AppendText("Wrong CompressionMethods" & xb360Generic.ComboBox1.Text.ToString)
Return False
End Select
End If
Return Nothing
End Function
#End Region
Private Sub Calculatefilecompression()
'/256MB
Dim L As String = FileSystem.FileLen(InputPath) '/File
strb.AppendLine()
strb.Append("Scan File......." & vbNewLine)
strb.Append(InputPath & vbNewLine)
strb.AppendLine()
strb.Append("Scan File.....done!" & vbNewLine)
strb.Append("Input:---> " & L & "(bytes)" & vbNewLine)
logMessage(strb.ToString)
strb.Clear()
If L < Mem Then
'/1Block
multiblocks = False
calculatedblocks = 1
Lastblocksize = L
Headersize = 0
blockstartpos = "&H0"
Dim positionheader As String = "&H0"
Dim sizeblock As UInteger = 0
Private _Maximum As Integer = 100
Private Percent As Integer = 0
Dim bytesRead As UInteger
Public Function ReadBufferBytes(Input As String, offset As String, sizeblock As UInteger) As Byte()
Dim buffer As Byte() = Nothing
Using fs As New FileStream(Input, FileMode.Open, FileAccess.Read, FileShare.Read)
strb.Append("Reade buffer bytes:---> " & sizeblock & "(bytes)" & vbNewLine)
logMessage(strb.ToString)
strb.Clear()
For i As Long = 0 To sizeblock
bytesRead = fs.Read(buffer, 0, CInt(sizeblock))
'/ update bei 1 Kb oder ende sizeblock
If i Mod 1024 = 0 Or i Mod sizeblock = 0 Then
fortschrittrb(i, sizeblock)
End If
Next
End Using
Return buffer
End Function
Private Function fortschrittrb(currentRead As UInteger, maxRead As UInteger)
Dim current As Integer
current = CDec(currentRead / maxRead * 100)
xb360Generic.RichTextBox2.Undo()'/Lazy clear last line from rtb
strb.Append("Progress:---> " & current & "%" & vbNewLine)
logMessage(strb.ToString)
strb.Clear()
Application.DoEvents()
Return Nothing
End Function
Private Sub StartCompression()
Dim messblocks As Integer = 1
If calculatedblocks = 1 Then sizeblock = Lastblocksize Else sizeblock = Mem
Dim offset As String = "&H0" '/Input file offset
Dim blockcouter As Integer = calculatedblocks
strb.AppendLine()
strb.Append("Compression start............" & blockstartpos & vbNewLine)
logMessage(strb.ToString)
strb.Clear()
Do Until endfile = True
Dim MemBuffer As Byte()
MemBuffer = ReadBufferBytes(InputPath, offset, sizeblock)
Dim compressedbuffer As Byte() = New Byte(MemBuffer.Length) {}
Dim compressedsize As UInteger = 0
strb.Append("Compress block (" & messblocks & ")" & vbNewLine)
logMessage(strb.ToString)
strb.Clear()
strb.Append("Compress buffer bytes:---> " & "0%" & vbNewLine)
logMessage(strb.ToString)
strb.Clear()
'/ senden an Compressor
For i As Long = 0 To sizeblock
compressedsize = lzo_compress(MemBuffer, MemBuffer.Length, compressedbuffer, compressedbuffer.Length, _
CUInt(CompressionMethodscompress(xb360Generic.ComboBox2.Text, True)), xb360Generic.levelNum.Value)
'/ update bei 1 Kb oder ende sizeblock
If i Mod 1024 = 0 Or i Mod sizeblock = 0 Then
fortschrittrb(i, sizeblock)
End If
Next
strb.Append("Compressed block size: " & compressedsize & "(bytes)" & vbNewLine)
strb.Append("Clear Memory buffer: " & MemBuffer.Length & "(bytes)" & vbNewLine)
strb.AppendLine()
logMessage(strb.ToString)
strb.Clear()
'/Lösche memory array von Compressor
Array.Clear(MemBuffer, 0, MemBuffer.Length)
'/check ob es was zu schreiben gibt
WriteBufferBytes(OutputPath, offset, compressedbuffer, compressedsize)
'/Nach schreiben array löschen
Array.Clear(compressedbuffer, 0, compressedbuffer.Length)
'/offset für writer setzen next block
blockstartpos = "&H" & CStr(Hex$(blockstartpos) + Hex$(compressedsize))
'/offset für next block berechnen
offset = offset + CUInt(sizeblock)
offset = "&H" & Hex$(offset)
'/block counten
messblocks += 1
blockcouter -= 1
'/Check ob letzter block erreicht ist und loop ende
If blockcouter = 0 Then Exit Do : endfile = True
'/ o vor 1 verhindert das sizeblock berechnet wird bei nur 1 block
If blockcouter = 1 Then sizeblock = Lastblocksize
Loop
End Sub
Private Function WriteBufferBytes(OutputPath As String, offset As String, compressedbuffer As Byte(), compressedsize As UInteger)
strb.Append("Write Compressed block: " & compressedsize & "(bytes)" & vbNewLine)
strb.AppendLine()
logMessage(strb.ToString)
strb.Clear()
Using fs As New FileStream(OutputPath, FileMode.OpenOrCreate, FileAccess.Write, FileShare.Write)
'/check ob file header da ist
Dim headerbuffer As Byte() = New Byte(Headersize - 1) {}
bw = New BinaryWriter(fs)
'/ check ob header benötigt wird für multi blocks durch boolen wert
If multiblocks = True Then
If positionheader = "&H0" Then
strb.Append("Create Big File Header: " & Headersize & "(bytes)" & vbNewLine)
strb.AppendLine()
logMessage(strb.ToString)
strb.Clear()
End If
'/schreibe die daten
fs.Seek(blockstartpos, SeekOrigin.Begin)
fs.Write(compressedbuffer, 0, compressedsize)
End Using
Return Nothing
End Function
Private Sub logMessage(ByVal message As String)
With xb360Generic.RichTextBox2
.AppendText(message)
.Refresh()
.ScrollToCaret()
End With
End Sub
Private Enum Endian
Little = 0
Big = 1
End Enum
Public Sub Write(value As Byte)
bw.Write(value)
End Sub
Public Sub WriteInt16(value As Short)
bw.Write(value)
End Sub
Public Sub WriteInt16(value As Short, type As Integer)
Dim buffer As Byte() = BitConverter.GetBytes(value)
If type = Endian.Big Then
Array.Reverse(buffer)
End If
bw.Write(value)
End Sub
Public Sub WriteUInt16(value As UShort)
bw.Write(value)
End Sub
Public Sub WriteUInt16(value As UShort, type As Integer)
Dim buffer As Byte() = BitConverter.GetBytes(value)
If type = Endian.Big Then
Array.Reverse(buffer)
End If
bw.Write(value)
End Sub
Public Sub WriteInt32(value As Integer)
Dim buffer As Byte() = BitConverter.GetBytes(value)
bw.Write(buffer)
End Sub
Private Sub WriteInt32(value As Integer, type As Integer)
Dim buffer As Byte() = BitConverter.GetBytes(value)
If type = Endian.Big Then
Array.Reverse(buffer)
End If
bw.Write(buffer)
End Sub
Public Sub WriteUInt32(value As UInteger)
Dim buffer As Byte() = BitConverter.GetBytes(value)
bw.Write(buffer)
End Sub
Private Sub WriteUInt32(value As UInteger, type As Integer)
Dim buffer As Byte() = BitConverter.GetBytes(value)
If type = Endian.Big Then
Array.Reverse(buffer)
End If
bw.Write(buffer)
End Sub
End Module
08-14-2013
Vulnavia
Re: Generic Xbox360 Compression Library
Does it actually work now?
Because i imported it into Delphi to use it for a Save which seem to be compressed, but all it does is copy IN to OUT, no matter what Compression Method i select
It acts like Memcopy, but does not compress anything and i fill the out buffer with $0 so its certainly the DLL that copy the stuff over.
Would be nice to get it work, i don't know if it will support the compression of the save, but well if it does not even compress anything, i don't even try to decompress something... lol