Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ListRecords() does not retrieve the entire record set in version 4.13 (v3.27 works fine) #10

Open
a-roussos opened this issue Jul 20, 2024 · 0 comments

Comments

@a-roussos
Copy link

a-roussos commented Jul 20, 2024

I'm trying to harvest records in oai_dc format from an OAI 2.0 repository by passing a specific setSpec to ListRecords().

The setSpec I'm using is 7375626A656374733D4F5A:4F5A3236, which corresponds to a setName of Subject = Ο: Οικονομική κρίση.

The set in question contains 233 records but I can only retrieve the last 33 when I use the latest version (4.13) of HTTP::OAI:

$ ./listrecords.pl https://anaktisis.uowm.gr/cgi/oai2 7375626A656374733D4F5A:4F5A3236
Using HTTP::OAI 4.13

2020-10-14T08:20:12Z oai:anaktisis.uowm.gr:10568
2020-10-12T07:03:10Z oai:anaktisis.uowm.gr:10572
2020-10-12T08:19:16Z oai:anaktisis.uowm.gr:10576
2020-10-16T08:19:48Z oai:anaktisis.uowm.gr:10589
2020-10-20T11:34:50Z oai:anaktisis.uowm.gr:10604
[...]
2022-07-06T08:00:52Z oai:anaktisis.uowm.gr:11753
2022-10-05T09:09:39Z oai:anaktisis.uowm.gr:12005
2022-11-25T08:00:25Z oai:anaktisis.uowm.gr:12047
2022-11-25T09:02:22Z oai:anaktisis.uowm.gr:12064
2022-11-30T09:02:28Z oai:anaktisis.uowm.gr:12066

Got 33 records

If I go back to version 3.27 (commit e151284), everything works fine (i.e. I can retrieve all 233 records):

$ ./listrecords.pl https://anaktisis.uowm.gr/cgi/oai2 7375626A656374733D4F5A:4F5A3236
Using HTTP::OAI 3.27

2015-04-01T07:24:06Z oai:anaktisis.uowm.gr:299
2015-04-01T12:30:38Z oai:anaktisis.uowm.gr:362
2015-11-11T09:50:10Z oai:anaktisis.uowm.gr:907
2015-05-19T10:09:23Z oai:anaktisis.uowm.gr:1793
2015-04-20T07:21:21Z oai:anaktisis.uowm.gr:2259
[...]
2022-07-06T08:00:52Z oai:anaktisis.uowm.gr:11753
2022-10-05T09:09:39Z oai:anaktisis.uowm.gr:12005
2022-11-25T08:00:25Z oai:anaktisis.uowm.gr:12047
2022-11-25T09:02:22Z oai:anaktisis.uowm.gr:12064
2022-11-30T09:02:28Z oai:anaktisis.uowm.gr:12066

Got 233 records

git bisect indicates that a8ae8b2 is the first bad commit (the latest good commit is 25f8de3).

Here's my code:

#!/usr/bin/perl

use strict;
use warnings;

use HTTP::OAI;

print 'Using HTTP::OAI ' . $HTTP::OAI::Harvester::VERSION . "\n\n";

my $harvester = HTTP::OAI::Harvester->new(
  baseURL => $ARGV[0]
);

my $records = $harvester->ListRecords(
  metadataPrefix => 'oai_dc',
  set            => $ARGV[1],
  from           => ''
);
die $records->message if $records->is_error;

my $count = 0;
while ( my $record = $records->next ) {
  $count++;
  print $record->datestamp . ' ' . $record->identifier . "\n";
}

print "\nGot " . $count . " records\n";
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant