Skip to content

Commit

Permalink
copy IndexFile from FastMail
Browse files Browse the repository at this point in the history
  • Loading branch information
@brong
brong committed Jan 29, 2018
1 parent 2519d4a commit d75589b
Showing 1 changed file with 150 additions and 27 deletions.
177 changes: 150 additions & 27 deletions Cyrus/IndexFile.pm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -210,12 +210,12 @@ SKIPPED VERSION 11 - Fastmail internal only
19: FirstExpunged time_t 4
20: LastCleanup time_t 4
21: HeaderFileCRC int32 4
22: SyncCRC int32 4
22: SyncCRCsBasic int32 4
23: RecentUid int32 4
24: RecentTime time_t 4
25: Spare0 int32 4
26: Spare1 int32 4
27: Spare2 int32 4
25: Pop3ShowAfter int32 4
26: QuotaAnnotUsed int32 4
27: SyncCRCsAnnot int32 4
28: HeaderCRC int32 4
Record:
Expand All @@ -237,6 +237,67 @@ SKIPPED VERSION 11 - Fastmail internal only
15: CacheCRC int32 4
16: RecordCRC int32 4
Version 14:
===========
Header:
0: Generation int32 4
1: Format int32 4
2: MinorVersion int32 4
3: StartOffset int32 4
4: RecordSize int32 4
5: Exists int32 4
6: LastAppenddate time_t 4
7: LastUid int32 4
8: QuotaUsed int64 8
9: Pop3LastLogin time_t 4
10: UidValidity int32 4
11: Deleted int32 4
12: Answered int32 4
13: Flagged int32 4
14: Options bitmap 4
15: LeakedCache int32 4
16: HighestModseq int64 8
17: DeletedModseq int64 8
18: Exists int32 4
19: FirstExpunged time_t 4
20: LastCleanup time_t 4
21: HeaderFileCRC int32 4
22: SyncCRC int32 4
23: RecentUid int32 4
24: RecentTime time_t 4
25: Pop3ShowAfter int32 4
26: QuotaAnnotUsed int32 4
27: SyncCRCsAnnot int32 4
28: Unseen int32 4
29: Spare0 int32 4
30: Spare1 int32 4
31: Spare2 int32 4
32: Spare3 int32 4
33: Spare4 int32 4
34: Spare5 int32 4
34: Spare6 int32 4
35: HeaderCRC int32 4
Record:
0: Uid int32 4
1: InternalDate time_t 4
2: SentDate time_t 4
3: Size int32 4
4: HeaderSize int32 4
5: GmTime time_t 4
6: CacheOffset int32 4
7: LastUpdated time_t 4
8: SystemFlags bitmap 4
9: UserFlags bitmap 16
10: ContentLines int32 4
11: CacheVersion int32 4
12: MessageGuid hex 20
13: Modseq int64 8
14: CID hex 8
15: CacheCRC int32 4
16: RecordCRC int32 4
=cut

# Set up header and record formatting information {{{
Expand Down Expand Up @@ -455,12 +516,74 @@ Exists int32 4
FirstExpunged time_t 4
LastCleanup time_t 4
HeaderFileCRC int32 4
SyncCRC int32 4
SyncCRCsBasic int32 4
RecentUid int32 4
RecentTime time_t 4
Pop3ShowAfter int32 4
QuotaAnnotUsed int32 4
SyncCRCsAnnot int32 4
HeaderCrc int32 4
EOF
RecordSize => 104, # defined in file too, check it!
_make_fields('Record', <<EOF),
Uid int32 4
InternalDate time_t 4
SentDate time_t 4
Size int32 4
HeaderSize int32 4
GmTime time_t 4
CacheOffset int32 4
LastUpdated time_t 4
SystemFlags bitmap 4
UserFlags bitmap 16
ContentLines int32 4
CacheVersion int32 4
MessageGuid hex 20
Modseq int64 8
CID hex 8
CacheCrc int32 4
RecordCrc int32 4
EOF
},
14 => {
HeaderSize => 160,
_make_fields('Header',<<EOF),
Generation int32 4
Format int32 4
MinorVersion int32 4
StartOffset int32 4
RecordSize int32 4
NumRecords int32 4
LastAppenddate time_t 4
LastUid int32 4
QuotaUsed int64 8
Pop3LastLogin time_t 4
UidValidity int32 4
Deleted int32 4
Answered int32 4
Flagged int32 4
Options bitmap 4
LeakedCache int32 4
HighestModseq int64 8
DeletedModseq int64 8
Exists int32 4
FirstExpunged time_t 4
LastCleanup time_t 4
HeaderFileCRC int32 4
SyncCRCsBasic int32 4
RecentUid int32 4
RecentTime time_t 4
Pop3ShowAfter int32 4
QuotaAnnotUsed int32 4
SyncCRCVersion int32 4
SyncCRCsAnnot int32 4
Unseen int32 4
Spare0 int32 4
Spare1 int32 4
Spare2 int32 4
Spare3 int32 4
Spare4 int32 4
Spare5 int32 4
Spare6 int32 4
HeaderCrc int32 4
EOF
RecordSize => 104, # defined in file too, check it!
Expand Down Expand Up @@ -552,7 +675,7 @@ sub _make_pack {
=item Cyrus::IndexFile->new($fh)
Build a new Cyrus::IndexFile object from a filehandle. The handle is not
Build a new Cyrus::IndexFile object from a filehandle. The handle is not
required to be seekable, so make sure you have rewound it before use.
seek($fh, 0, 0);
Expand All @@ -561,7 +684,7 @@ required to be seekable, so make sure you have rewound it before use.
This function reads the header from the file and returns a Cyrus::IndexFile
object. The filehandle will be pointing at the start of the first record.
If there is a problem, then the position of the filehandle is undefined
If there is a problem, then the position of the filehandle is undefined
(though probably either at 12 bytes or the end of the header) and the
function will "die".
Expand All @@ -580,7 +703,7 @@ sub new {

# read initial header information to determine version
my $read = sysread($handle, $buf, 12);
die "Unable to read header information\n"
die "Unable to read header information\n"
unless $read == 12;

# version is always at this offset!
Expand All @@ -592,9 +715,9 @@ sub new {

# read the rest of the header (length depends on version)
sysread($handle, $buf, $frm->{HeaderSize} - 12, 12);
my $Self = bless {
my $Self = bless {
@_,
version => $version,
version => $version,
handle => $handle,
format => $frm,
rawheader => $buf,
Expand Down Expand Up @@ -630,7 +753,7 @@ sub new_file {
$fh = IO::File::fcntl->new($filename, '+<', @$lockopts)
|| die "Can't open $filename for locked read: $!";
} else {
$fh = IO::File->new("< $filename")
$fh = IO::File->new("< $filename")
|| die "Can't open $filename for read: $!";
}

Expand All @@ -650,10 +773,10 @@ sub new_empty {
my $version = shift;

# check that the version is supported
my $frm = $VersionFormats->{$version}
my $frm = $VersionFormats->{$version}
|| die "unknown version $version";

my $Self = bless {
my $Self = bless {
@_,
version => $version,
format => $frm,
Expand Down Expand Up @@ -772,7 +895,7 @@ sub header_copy {

=item $index->reset($num)
Deletes the cached 'current record' and seeks back to the given record
Deletes the cached 'current record' and seeks back to the given record
number, or the end of the header (record 0) if no number given.
Requires the input filehandle to be seekable.
Expand All @@ -783,15 +906,15 @@ sub reset {
my $Self = shift;
my $num = shift || 0;

my $NumRecords = $Self->{header}{MinorVersion} < 12 ?
$Self->{header}{Exists} : $Self->{header}{NumRecords};
my $NumRecords = $Self->{header}{MinorVersion} < 12 ?
$Self->{header}{Exists} : $Self->{header}{NumRecords};

die "Invalid record $num (must be >= 0 and <= $NumRecords"
unless ($num >= 0 and $num <= $NumRecords);

my $HeaderSize = $Self->{format}{HeaderSize};
my $RecordSize = $Self->{format}{RecordSize};

sysseek($Self->{handle}, $HeaderSize + ($num * $RecordSize), 0)
|| die "unable to seek on this filehandle";

Expand Down Expand Up @@ -851,8 +974,8 @@ sub next_record_raw {
delete $Self->{checksum_failure};

# use direct access for speed
my $NumRecords = $Self->{header}{MinorVersion} < 12 ?
$Self->{header}{Exists} : $Self->{header}{NumRecords};
my $NumRecords = $Self->{header}{MinorVersion} < 12 ?
$Self->{header}{Exists} : $Self->{header}{NumRecords};
my $RecordSize = $Self->{header}{RecordSize};

return undef unless $RecordSize;
Expand Down Expand Up @@ -980,7 +1103,7 @@ sub record_raw {

=item $index->field_number($Field)
Return the field number in a record array for the named field, or die
Return the field number in a record array for the named field, or die
if there isn't one.
=cut
Expand Down Expand Up @@ -1022,9 +1145,9 @@ Also seeks back to the header and rewrites it with exists incremented by one.
sub append_record {
my $Self = shift;
my $record = shift;
my $NumRecords = $Self->{header}{MinorVersion} < 12 ?
$Self->{header}{Exists} : $Self->{header}{NumRecords};

my $NumRecords = $Self->{header}{MinorVersion} < 12 ?
$Self->{header}{Exists} : $Self->{header}{NumRecords};

$Self->reset($NumRecords);
$Self->write_record($Self->{handle}, $record);
Expand Down Expand Up @@ -1163,9 +1286,9 @@ sub merge_indexes {
=item $index->record_undump()
Dump the headers and records in either space separated fields or named lines with a blank line between for long.
Dump the headers and records in either space separated fields or named lines with a blank line between for long.
The "undump" option is able to parse the space separated format, allowing pipe to a standard unix tool to
The "undump" option is able to parse the space separated format, allowing pipe to a standard unix tool to
process the records, and then re-parse them back into a binary index file.
=cut
Expand Down Expand Up @@ -1446,7 +1569,7 @@ sub _record_h2a {

=item AUTHOR AND COPYRIGHT
Bron Gondwana <brong@fastmail.fm> - Copyright 2008 FastMail
Bron Gondwana <brong@fastmailteam.com> - Copyright 2018 FastMail
Licenced under the same terms as Cyrus IMAPd.
Expand Down

0 comments on commit d75589b

Please sign in to comment.