forked from nnguyen/google-enterprise-connector-manager
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathRELEASE_NOTES
1940 lines (1553 loc) · 88.5 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Google Search Appliance Connector Manager
Release Notes
This document contains the release notes for Google Search Appliance
Connector Manager. The following sections describe the release in
detail and provide information that supplements the main documentation.
See the Issues Tab on the Code Site for the current list of known
issues and work-arounds.
Web Site: http://code.google.com/p/google-enterprise-connector-manager/
Release 3.2.10, 24 March 2015
=============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in
the previous release. Most users do not need to upgrade. Connector
developers and customers manually deploying connectors with Tomcat 8
are encouraged to upgrade.
Issues Fixed Since 3.2.6
------------------------
18248756 - Increase the default traversal batch timeout from 30 minutes
to 2 hours.
Note: The default value of the traversal.time.limit property
is set in applicationContext.xml, but it may be overridden in
applicationContext.properties.
18969469 - NDC and MDC logging context strings were empty under Tomcat 8.
Note: When manually upgrading Connector Manager, this fix
requires the new META-INF/context.xml file from the
connector-manager.war file.
Version Compatibility
---------------------
This version of the Connector Manager requires Java 6 JRE or newer.
For connector developers, Subversion 1.8 and Java 8 are supported.
Release 3.2.6, 25 April 2014
==============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of previous releases are encouraged to upgrade.
Issues Fixed Since 3.2.4
------------------------
14303102 - The DatabaseConnectionPool utility class did not detect
dead connections to SQL Server. This class now requires a
JDBC driver that supports the isValid method of
java.sql.Connection, part of the JDBC 4.0 specification in
Java 6. The embedded H2 database has a compliant driver.
For Oracle or SQL Server, a compliant driver must be
configured. This class is used by the AD Groups connector
starting with version 3.2.6, and by the Lotus Notes
connector since version 2.8.4.
Version Compatibility
---------------------
This version of the Connector Manager requires Java 6 JRE or newer.
The embedded JUnit JAR file was upgraded from version 3 to version 4.8.2.
The embedded EasyMock JAR files were upgraded from version 3.0 to version
3.2.
Deprecated Features
-------------------
The following util package feature has been deprecated and is
targeted for removal in a future release.
* The IOExceptionHelper class. Replaced in Java 6 by the
IOException(String, Throwable) constructor.
Release 3.2.4, 16 January 2014
==============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of previous releases are encouraged to upgrade.
Issues Fixed Since 3.2.2
------------------------
11718150 - GetDocumentContent was returning the google:aclinheritfrom:
fragment property as metadata. This property is intended for
internal use, and should not have been provided to the GSA.
12176042 - EncryptedPropertyPlaceholderConfigurer should not try to
decrypt the empty string. This could prevent Tomcat startup
because the default H2 database password is empty.
12576455 - Add support for crawl-immediately and crawl-once feed record
properties. You can use these properties with a document filter,
such as the AddPropertyFilter.
Version Compatibility
---------------------
This version of the Connector Manager requires Java 6 JRE or newer.
Removed Features
-----------------
Support for the gsa.admin.requiresPrefix configuration property has been
removed. If the property is specified, it will be ignored.
Only GSA versions prior to v5.0.4 used this feature, and they were never
supported by the connectors.
Release 3.2.2, 24 October 2013
==============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of previous releases are encouraged to upgrade.
Summary of Changes
------------------
Fix issue 7766078: Reduce the default feed backlog configuration parameters.
The default feed.backlog.ceiling dropped from 10000 to 4000.
The default feed.backlog.floor remains 1000.
The default feed.backlog.interval dropped from 900 to 120
seconds.
Fix issue 7881428: Support more unregistered Office media types from SharePoint.
application/vnd.ms-excel.macroEnabled.12
application/vnd.ms-powerpoint.macroEnabled.12
application/vnd.ms-word.macroEnabled.12
Fix issue 9534168: Protect invalid Traversal Rate configuration. Negative
values would confuse the HostLoadManager's traversal rate
calculations.
Fix issue 10253530: H2 1.2 Database corruption. This release updates the
embedded H2 Database to version 1.3.173. For more
information, see http://www.h2database.com .
Fix issue 10373520: Roles were misinterpreted in DENY principals. This problem
was introduced by the fix to issue 10263958 in 3.2.0.
Fix issue 10404326: A NullPointerException would be thrown out of the
SkipDocumentFilter if the source document supplied
a null value for the property being checked.
Fix issue 10845203: Remove URL-encoding from document IDs for delete URLs
in the diffing package. The DeleteDocumentHandle class
was violating the contract for the DocumentHandle
interface, where the value returned by getDocumentId()
"must match the value returned by calling
getDocument().findProperty(PROPNAME_DOCID)."
Fix issue 11341445: A diffing connector could get into a loop deleting all
of the snapshot files on each pass, and therefore
feeding everything to the GSA for indexing on each
pass. At least the two most recent snapshot files are
now saved.
Version Compatibility
---------------------
This version of the Connector Manager requires Java 6 JRE or newer.
Diffing connectors that use non-URL safe docids and expect the
DeleteDocumentHandle class to URL encode the docids will have to modified
to use consistent URL-encoding in the connector or to use URL safe docids.
Deprecated Features
-------------------
The following SPI features have been deprecated and are targeted for
removal in a future release. These features were never fully implemented
or functional in any version of the Connector Manager or GSA.
* The SPI LocalDocumentStore, which would have been available via
the SPI ConnectorPersistentStore interface. The LocalDocumentStore
was never fully implemented and ConnectorPersistentStore has always
returned null from getLocalDocumentStore(). The following SPI constants
associated with the LocalDocumentStore have also been deprecated:
* SpiConstants.PERSISTABLE_ATTRIBUTES
* SpiConstants.PROPNAME_MANAGER_SHOULD_PERSIST
* SpiConstants.PROPNAME_CONNECTOR_INSTANCE
* SpiConstants.PROPNAME_CONNECTOR_TYPE
* SpiConstants.PROPNAME_PRIMARY_FOLDER
* SpiConstants.PROPNAME_TIMESTAMP
* SpiConstants.PROPNAME_MESSAGE
* SpiConstants.PROPNAME_SNAPSHOT
* SpiConstants.PROPNAME_CONTAINER
* SpiConstants.PROPNAME_PERSISTED_CUSTOMDATA_1
* SpiConstants.PROPNAME_PERSISTED_CUSTOMDATA_2
* SpiConstants.PROPNAME_FEEDID
The following diffing package feature has been deprecated and is
targeted for removal in a future release.
* SnapshotStore.getOldestSnapsotToKeep [the method name is misspelled]
Release 3.2.0, 13 August 2013
=============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of previous releases are encouraged to upgrade.
Summary of Changes
------------------
Fix issue 8161087: Lister/Retriever connectors may now supply metadata at
crawl-time, rather than feed-time, for GSA 7.2 and newer.
This avoids issue 6781122, wherein the internal metadata
extracted from documents is overwritten by the external
metadata when the document is sent again in a feed.
Fix issue 8256145: Passwords longer than 9 characters were incorrectly
encrypted when using the command-line EncryptPassword
tool. This typically manifested as failures logging
into content reposistories for the connectors.
Fix issue 9415184: MimeTypeDetector would inadvertently attempt to open
some of the supplied files when it was attempting to
determine the MIME type by filename extension only.
Since the supplied names were rarely local files,
the MimeTypeDetector would quickly become backlogged
on blocked file I/O. The use of the underlying third
party MimeUtil library was altered to ensure that no
attempt would be made to open the named file.
Fix issue 10263958: ACL Principals with a peeker role were not removed from
the feed. This would inadvertently grant full read
access to users and groups that had only peeker access.
This first appeared in Connector Manager version 3.0.0,
but only if used with GSA version 7.0 and newer. This
affected the SharePoint connector and any third-party
connectors that implemented the peeker role.
Version Compatibility
---------------------
This version of the Connector Manager requires Java 6 JRE or newer.
The use of a Java 5 runtime is no longer supported.
Deprecated Features
-------------------
The following SPI features have been deprecated and are targeted for
removal in a future release. These features were never fully implemented
or functional in any version of the Connector Manager or GSA.
* Roles in ACLs ('reader', 'writer', 'owner', 'peeker').
This includes the SpiConstants.RoleType enum and the associated Role
Properties prefixes SpiConstants.GROUP_ROLES_PROPNAME_PREFIX, and
SpiConstants.USER_ROLES_PROPNAME_PREFIX. Roles were never used, and
were stripped from ACL entries if supplied.
* SpiConstants.PROPNAME_CONTENTURL. This was never used.
* SpiConstants.PROPNAME_SECURITYTOKEN. This was never used.
Release 3.0.8, 08 May 2013
==========================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of previous releases are encouraged to upgrade.
Summary of Changes
------------------
Fix Issue 6513938: Allow the connector to specify a document's content encoding
with a google:contentencoding SPI property. This allows the
connector to return content that has already been encoded in
one of the supported content encodings (e.g. base64binary).
Fix issue 6734754: GSA cannot handle null Schedule. The GSA, while claiming
to support null or empty schedules, actually does not.
If a connector does not have a schedule, getConnectorStatus
servlet will supply a schedule with default load, retryDelay,
and interval, but disabled.
Fix issue 7040140: The GetDocumentContent servlet should provide Last-Modified
date in HttpResponse.
Fix issue 7928861: Avoid unordered snapshots in the snapshot files of Diffing
Connectors.
Fix Issue 8078850: Add a configuration option to AclPropertyFilter to specify
a domain for the users in Prinicipals.
Fix issue 8207127: Reduce the memory footprint of DocumentHandle
serialization for Diffing connectors.
Fix issue 8237465: Connector instantiation would fail with non-ASCII
characters in the advanced properties XML.
Fix Issue 8394155: SkipDocumentFilter and ModifyPropertyFilter would throw
NullPointerException if the target property has a null
string value.
Fix Issue 8461333: Add google:authmethod. Added an optional, single-valued
string SPI property, google:authmethod, to specify the
authentication method. Users can override the default
(httpbasic) by adding an AddPropertyFilter to set
google:authmethod in a connector's advanced configuration:
<bean class="com.google.enterprise.connector.util.filter.AddPropertyFilter">
<property name="propertyName" value="google:authmethod"/>
<property name="propertyValue" value="ntlm"/>
<property name="overwrite" value="true"/>
</bean>
Fix issue 8592252: GetDocumentContent now optionally supplies a Content-Length
header in its servlet response. This adds a new SPI property,
google:contentlength, that the connector may supply,
which specifies the length of the document content, in bytes.
If the google:contentLength is supplied by the connector,
that value is used in the Content-Length HTTP header in
the servlet response. This feature only applies to
connectors with a Retriever implementation.
Release 3.0.4, 19 November 2012
===============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of previous releases are encouraged to upgrade.
Summary of Changes
------------------
Fix Issue 7341404: Diffing connectors ignored Retry Delay setting. Diffing
connectors will now schedule incremental traversals
according to the Retry Delay setting specified on the
connector configuration page.
Fix Issue 7364602: Inaccessible LDAP server kills Connector Manager.
Loading the connector instances asynchronously now moves
their creation out of the Connector Manager servlet StartUp
code path, so an individual connector instance timing out
will not cause the Connector Manager to fail start-up.
Fix Issue 7409537: Support more unregistered Microsoft Office 2007 media types:
"application/vnd.ms-excel.12",
"application/vnd.ms-powerpoint.12",
"application/vnd.ms-word.12".
Fix Issue 7554816: Support DENY with flattened ACLs in SharePoint connector.
This change adds a TraversalContext.supportsDenyAcls method
that is distinct from the supportsInheritedAcls method.
Previously DENY ACL support was implied if inherited ACLs
were supported. This change allows DENY ACLs to be sent
to GSA 7.x, even if feed.disable.inherited.acls property
is set.
Fix Issue 7584684: GsaFeedConnection needs to invalidate the cached DTD when
switching GSAs. If a running connector manager is
reregistered with a different version of the GSA, it
would continue to assume the capabilities of the original
GSA version, based on an inspection of the feed DTD.
Release 3.0.2, 17 October 2012
==============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of previous releases are encouraged to upgrade.
Summary of Changes
------------------
Fix Issue 7166051: Deny HTTP HEAD requests from legacy authorization, which
would always permit access to secure documents.
Fix Issues 7312517, 7343328: Multiple cases of NullPointerException
thrown from the diffing connector SnapshotStore.
Fix Issue 7343330: BasicChecksumGenerator.getDigest() throws NullPointerException.
Fix Issue 7369686: Diffing connectors ignore the configured schedule intervals.
Diffing connectors would continue to run, consuming
resources and traversing the repositories when outside
of a scheduled traversal interval.
Release 3.0.0, 13 September 2012
================================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of previous releases are encouraged to upgrade.
Summary of Changes
------------------
Fix Issue 4765123: Set a logging context in the diffing package. This provides
more informative logging from the various diffing connector
threads.
Fix Issue 6299907: Connector Manager restart resets the logging level.
Fix Issue 6441063: Set the feed.contenturl.prefix advanced configuration
property when Connector Manager is registered with the GSA.
Fix Issue 6449767: Initial diffing connector snapshot fails in Java 7.
Fix Issue 6861210: Diffing fileSystemMonitorsByName is not cleared when
DocumentSnapshotRepositoryMonitorManagerImpl is stopped.
Fix Issue 6867242: Diffing connector may leak DocumentSnapshotRepositoryMonitor
threads when toggling "Disable traversal" check box and the
connector may go into inconsistent state.
Fix Issue 6942176: CheckpointAndChangeQueue throws NullPointerException.
Fix Issue 6996468: Provide advanced configuration option to not use inherited
ACLs with GSA v7.0. This is provided as a workaround for
GSA issue 6969557, where inherited ACLs do not work correctly
with Distributed Crawl and Serve.
Release 2.8.10, 28 November 2012
================================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of previous releases are encouraged to review
the changes below to determine whether to upgrade.
Summary of Changes
------------------
Fix Issue 4765123: Set a logging context in the diffing package. This provides
more informative logging from the various diffing connector
threads.
Fix Issue 5599305: Retry Connector startup if instantiation fails. The
FileSystem Connector fails Connector bean instantiation if
the file share is off-line. The other connectors that use
the Diffing package (LDAP and Database) can suffer similar
failures. This fix allows a failed Connector instantiation
to be retried after a period, in hopes that any transient
errors may have been corrected.
Fix Issue 6299907: Connector Manager restart resets the logging level.
Fix Issue 6861210: Diffing fileSystemMonitorsByName is not cleared when
DocumentSnapshotRepositoryMonitorManagerImpl is stopped.
Fix Issue 6867242: Diffing connector may leak DocumentSnapshotRepositoryMonitor
threads when toggling "Disable traversal" check box and the
connector may go into inconsistent state.
Fix Issue 6942176: CheckpointAndChangeQueue throws NullPointerException.
Fix Issues 7312517, 7343328: Multiple cases of NullPointerException
thrown from the diffing connector SnapshotStore.
Fix Issue 7341404: Diffing connectors ignored Retry Delay setting. Diffing
connectors will now schedule incremental traversals
according to the Retry Delay setting specified on the
connector configuration page.
Fix Issue 7343330: BasicChecksumGenerator.getDigest() throws
NullPointerException.
Fix Issue 7364602: Inaccessible LDAP server kills Connector Manager.
Loading the connector instances asynchronously now moves
their creation out of the Connector Manager servlet StartUp
code path, so an individual connector instance timing out
will not cause the Connector Manager to fail start-up.
Fix Issue 7369686: Diffing connectors ignore the configured schedule intervals.
Diffing connectors would continue to run, consuming
resources and traversing the repositories when outside
of a scheduled traversal interval.
Fix Issue 7409537: Support more unregistered Microsoft Office 2007 media types:
"application/vnd.ms-excel.12",
"application/vnd.ms-powerpoint.12",
"application/vnd.ms-word.12".
Release 2.8.6, 5 May 2012
=========================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of previous releases are encouraged to review
the changes below to determine whether to upgrade.
Summary of Changes
------------------
* Fix Issue 6305209 - Text conversion fails on PDF files when skipping
the content. Handle PDF documents that are zero-length or too long more
gracefully. Rather than skip the document entirely, feed a stub
document with just the document's title, if available.
* Fix file system connector code site issue 32 - Initial snapshot fails
in Java 7 with error "two snapshots with the same number". Note that
Java 7 is not officially supported.
* Differentiate between no password and empty-string password in the
user authentication servlet.
* Remove the google:feedid property from records in the feeds.
Release 2.8.4, 23 February 2012
===============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of the previous release are encouraged to upgrade.
Summary of Changes
------------------
* Fix Issue 5973714: Exceptions thrown while prefetching the Authorization
Manager on connector startup would leave the connector instance in an
inconsistent state.
* Fix Issue 5723358 - Escape special characters in user and group names
returned in an Authentication response. User or group names that contained
certain characters that have special meaning in XML syntax would cause
failures reading the Authentication response.
* Fix Issues 5370948 and 5481676 - Better recovery from FeedExceptions.
If submitting a feed to the GSA fails for some reason, the Connector
Manager retrys the feed after 15 minutes. But first, the Connector
Manager would test the GSA to verify that it is accepting feeds.
Unfortunately, that test would actually kill a functioning GSA
feedergate, disabling feeds for a short period of time while it
restarts. Effectively, the feed problem recovery strategy would
kill feeds every 15 minutes. This problem only affects GSA version
6.12. This fix avoids the problem, using a slightly different strategy
to check for GSA feed availability.
* Address Issue 5382030 - If Flexible Authorization is misconfigured
to use connector authorization with a credential group which has no
authentication rules defined, the GSA sends a null Identity to the
Connector Manager during Authorization. This was handled poorly
by most Connectors. Although Issue 5382030 is actually a problem
with the Security Manager, the Connector Manager now considers a null
Identity to be an error, and returns an error status code to the
GSA.
* Adds rudimentary GData configuration for Connectors. The new
googleFeedHost property supplied to Connectors may be used to
access the GData interface on the GSA. This should be considered,
at best, a temporary solution. This change also removes the
googleWorkDir and googleConnectorWorkDir properties from the saved
properties files, to avoid problems when moving connector instances
to a different directory. The properties still appear in the Properties
objects in the SPI.
* Adds several improvements to the Document Filters.
The ModifyPropertyFilter adds support for modifying the CONTENT
property of text documents (determined according to MIME type).
A SkipDocumentFilter can force a document to be skipped (or not)
based upon the presence/abscence of a specific Property, or based
upon a match on one of the values of that property.
The JavaDoc documentation for the Document Filters has been
improved, including example configurations.
* Various improvements in diagnostic logging.
* Fix Issues 233, 5028655, 6019938 - Fix logic bug in diffing where
recovery-files' age comparison was broken. This could lead to the
connector resending the same files again after Tomcat was restarted.
* Fix Issue 232 - A small memory leak in ThreadPool would leak
QueryTraversers (and all the objects they held).
Version Compatibility
=====================
The diffing library has a change effecting diffing connectors (File System,
LDAP, and Database). The method for assigning file name extensions to
recovery files has changed. This change causes no issues migrating forward to
this release, but reverting to an earlier release after running 2.8.4 requires
diffing connectors to be reset.
Release 2.8.2, 10 October 2011
==============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of the previous release are encouraged to upgrade.
Summary of Changes
------------------
* Enable editing of Connector Advanced configuration XML from the
GSA Admin console.
* Adds support for Flexible Authorization for Connectors.
* Improves MIME type recognition for many Microsoft Office file formats
when using the third-party mime-util detector.
* Reduces the likelihood of search result authorization timeouts.
Release 2.8.0, 08 July 2011
===========================
Introduction
------------
This release has significant infrastructure changes, fixes several
problems, and adds several new utility classes to the connector SPI
for the benefit of connector developers.
Summary of Changes
------------------
* Issue 104 - Servlet to dynamically change logging levels. New servlets
getConnectorLogLevel, setConnectorLogLevel, getFeedLogLevel and
setFeedLogLevel allow the connector administrator to adjust the connector
and feed logging verbosity without shutting down the connector.
These are especially useful in conjunction with the existing
getConnectorLogs and getFeedLogs servlets.
* Issue 168 - Make Base64 encode/decode available to the connector developer.
Base64, Base64FilterInputStream, and Base64ChecksumGenerator are several
of the new utility classes made available to connector developers.
* Issue 199 - SPI enhancement to expose JDBC to the connector developer.
Enabled by the new ConnectorPersistentStore SPI interface. Connectors
that wish to be given access to the JDBC DataSource should implement the
ConnectorPersistentStoreAware interface in their Connector implementation.
Given the DataSource, the connector developer may also take advantage
of several new database access utility classes, such as JdbcDatabase,
DatabaseConnectionPool, and DatabaseResourceBundle.
Note that ConnectorPersistentStore.getLocalDocumentStore() is disabled
in this release.
* Fixed Issue 4062256 - Failure to delete snapshot files would throw
IllegalStateException. This affected the File System, LDAP, and
Database Connectors.
* Fix Issue 4524076 - Backward compatibility issue in diffing connectors
for recovery files. This affected the File System, LDAP, and
Database Connectors.
* Fix Issues 4581062, 4613042 - Add configurable diffing connector delay
interval after each scan: 'introduceDelayAfterEachScan'. This should
relieve some of the continuous file system scanning behaviour in the
File System Connector.
* The SPI AuthenticationManager and AuthenticationResponse classes have
been enhanced to allow the connector to return repository local groups
for a user.
* A new pagerank document property is now supported.
The SpiConstants.PROPNAME_PAGERANK property allows the connector to
recommend a pagerank (0-100) for the document if it matches queries.
For more information on pagerank see:
http://code.google.com/apis/searchappliance/documentation/610/feedsguide.html#defining_the_xml
* Fixed a minor problem that prevented connectors from running in the
JBoss Application Server. See the JBoss deployment wiki page:
http://code.google.com/p/google-enterprise-connector-manager/wiki/JBossCM
* Added support for these Microsoft Office 2007 and later media types:
- application/vnd.ms-outlook
- application/vnd.ms-excel.sheet.12
- application/vnd.ms-powerpoint.presentation.12
- application/vnd.ms-word.document.12
* Added support for Secure Socket Layer (SSL) feeds to the Google Search
Appliance. At the present time, SSL feeds must be manually configured.
For additional details, see the Advanced Configuration wiki page:
http://code.google.com/p/google-enterprise-connector-manager/wiki/AdvancedConfiguration
* New Document Filters utility package additions to the SPI for use by
connector developers, connector administrators, and systems integrators.
Document filters act to transform their source Document's Properties.
Document filters can add, remove, or modify a document's properties,
including the document content. Properties in which the filter has
no interest are passed through unmodified. A document filter might
even throw a SkippedDocumentException to prevent a document from being
fed to the Google Search Appliance.
Multiple document filters may be chained together, forming
a transformational document processing pipeline. Similar to a
Unix command pipeline, the filters are linked together, each using
the previous one as its source Document.
For more information see the Document Filters wiki page:
http://code.google.com/p/google-enterprise-connector-manager/wiki/DocumentFilters
* New additions to the SPI:
- ConnectorPersistentStore - Provides access to the LocalDatabase
- ConnectorPersistentStoreAware - Advertises that the Connector
wishes access to the LocalDatabase
- DatabaseResourceBundle - Vendor-specific SQL language translations
- LocalDatabase - Provides access to the configured JDBC DataSource and
DatabaseResourceBundles
- SpiConstants.PROPNAME_PAGERANK
For additional information, please refer to the JavaDoc at:
http://google-enterprise-connector-manager.googlecode.com/svn/docs/javadoc/2.8.0/index.html
* New utility package additions to the SPI for use by connector developers:
(available in package com.google.enterprise.connector.util)
- Base64 - Base64 encode/decode utility
- Base64DecoderException
- Base64FilterInputStream - InputStream filter that Base64 encodes
data read from its input
- ChecksumGenerator - Interface for checksum generators
- BasicChecksumGenerator - Generates MD2, MD5, SHA-1, SHA-256, SHA-384
and SHA-512 message digest checksums of data from an InputStream
- Base64ChecksumGenerator - Derived from BasicChecksumGenerator, but
returns Base64 encoded checksums
- Clock - a interface for getting the time; useful to replace for testing
- SystemClock - a Clock implementation using System.getCurrentTimeMillis()
- EofFilterInputStream - InputStream filter that avoids a read at
end-of-file problem with Apache Commons IO AutoCloseInputStream
- IOExceptionHelper - creates IOExceptions with a root cause on Java 5
- UniqueIdGenerator - Interface for producing unique IDs
- UuidGenerator - UniqueIdGenerator implementation based on UUID
- XmlParseUtil - utility methods for parsing XML data
- SAXParseErrorHandler
For additional information, please refer to the JavaDoc at:
http://google-enterprise-connector-manager.googlecode.com/svn/docs/javadoc/2.8.0/index.html
* New utility database package additions to the SPI for use by connector
developers:
(available in package com.google.enterprise.connector.util.database)
- JdbcDatabase - database info, utilities for creating and maintaining
database tables for connector instances.
- DatabaseConnectionPool - a pool of connections to the JDBC DataSource
- DatabasePropertyResourceBundle - DatabaseResourceBundles implemented
as properties files
- DatabaseResourceBundleManager - loads DatabaseResourceBundles
For additional information, please refer to the JavaDoc at:
http://google-enterprise-connector-manager.googlecode.com/svn/docs/javadoc/2.8.0/index.html
* New utility diffing package addition to the SPI provides a snapshot
diffing connector framework for use by connector developers:
(available in package com.google.enterprise.connector.util.diffing)
- Change, ChangeQueue, ChangeSource
- CheckpointAndChange, CheckpointAndChangeQueue
- DeleteDocumentHandle, DeleteDocumentHandleFactory
- DocumentHandle, DocumentHandleFactory
- DiffingConnector, DiffingConnectorTraversalManager
- DiffingConnectorCheckpoint, DiffingConnectorDocumentList
- DocIdUtil
- FilterReason
- GenericDocument
- DocumentSink, LoggingDocumentSink
- DocumentSnapshot, DocumentSnapshotFactory
- DocumentSnapshotRepositoryMonitor
- DocumentSnapshotRepositoryMonitorManager
- DocumentSnapshotRepositoryMonitorManagerImpl
- MonitorCheckpoint
- SnapshotRepository, SnapshotRepositoryRuntimeException
- SnapshotStore, SnapshotStoreException
- SnapshotReader, SnapshotReaderException
- SnapshotWriter, SnapshotWriterException
- TraversalContextManager
For additional information, please refer to the JavaDoc at:
http://google-enterprise-connector-manager.googlecode.com/svn/docs/javadoc/2.8.0/index.html
* New utility database and diffing testing packages additions to the SPI
provides test classes for use by connector developers:
(available in package com.google.enterprise.connector.util.database.testing)
- TestJdbcDatabase
- TestLocalDatabase
- TestResourceClassLoader
(available in package com.google.enterprise.connector.util.diffing.testing)
- FakeDocumentSnapshotRepositoryMonitorManager
- FakeTraversalContext
- TestDirectoryManager
* The Connector Manager now ships with several new third party JARs.
The connector developer may find these functionally useful, however
they should note that these are now distributed with the Connector
Manager and the connectors should take care not to replace them
with older or incompatible versions.
- commons-cli.jar v1.2 http://commons.apache.org/cli
- eproperties.jar v1.1.0 http://code.google.com/p/eproperties
- h2.jar v1.2.147 http://www.h2database.com
Release 2.6.10, 04 February 2011
================================
Introduction
------------
This is an internal release for the Connector Manager on-board the GSA,
not for general use.
Release 2.6.6, 7 December 2010
===============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. New additions have been made to the connector SPI.
Users of the previous release are encouraged to upgrade.
Summary of Changes
------------------
* Issue 217 - Schedule intervals that include midnight were treated as
empty and ignored.
* Issue 224 - Fixes a potential loss of information about exceptions in
the connector logs.
* Issue 225 - Fixes a series of problems with the ImportExport utility.
* Issue 227 - Use ' instead of ' when escaping single quotes,
for HTML compatibility. The use of ' could lead to errors when
configuring connector instances using Internet Explorer.
* Improved log messages when free memory is low, when the feeds are
paused due to a backlog on the GSA, and when constructing a new
connector instance throws an exception while starting a new traversal
batch.
* New additions to the SPI:
o SpiConstants.RESERVED_PROPNAME_PREFIX
o SpiConstants.PROPNAME_FOLDER
o SpiConstants.PROPNAME_LOCK
o UrlValidator class
o UrlValidatorException class
The UrlValidator class is in a new com.google.enterprise.connector.util
package. This package will be used for utility classes that are not
part of or related to the spi package, but which connector
implementers might find useful.
Release 2.6.4, 16 September 2010
================================
Introduction
------------
This is an internal release for the Connector Manager on-board the GSA,
not for general use.
Summary of Changes
------------------
* Servlet access to the Connector Manager on-board the GSA has been
largely locked down. The getConfiguration and getConnectorLogs
servlets remain accessible for the benefit of connector administrators
and support personnel.
Release 2.6.2, 09 September 2010
================================
Introduction
------------
This is an internal release for the Connector Manager on-board the GSA,
not for general use.
Release 2.6.0, 14 June 2010
============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of the previous release are encouraged to upgrade.
Summary of Changes
------------------
* Issue 189 - Fixes a problem with the HostLoadManager that would
impose artificial traversal delays. The delays, although short,
would occur frequently. This fix allows the Connector Manager to
more accurately adhere to the configured documents per minute
traversal rate.
* Issue 194 - Enhances the HostLoadManager to more accurately adhere
to the configured documents per minute traversal rate; while allowing
the Connector to occasionally exceed that rate to improve efficiency.
* Issue 220 - Fixes the representation of multiple-valued metadata supplied
by the Connector as it is fed to the Google Search Appliance. This
corrects a problem with parametric navigation.
* Fixed a problem where restarting a traversal from the beginning
of a Connector's Repository or changing a Connector's traversal
schedule might not take immediate effect.
* Added a 'traversal.enabled' property that may be set in the Connector
Manager's applicationContext.properties is used to enable or disable
Traversals and Feeds for all Connector instances in the Connector
Manager. Disabling Traversal would be desirable if configuring a
Connector Manager deployment that only authorizes search results.
This feature is designed for turning off traversal for replica
Connector Managers in a clustered, load-balanced, or fail-over
environment. Traversals are enabled by default.
* Added an EncryptPassword command line utility that can be used by
an administrator to encrypt passwords that will be manually added to
Connector Manager or Connector properties files. For details, see
the EncryptPassword wiki page at:
http://code.google.com/p/google-enterprise-connector-manager/wiki/EncryptPassword
* Enhanced support for the Apache JULI LogManager and FileHandler.
* Corrected a minor issue with regards to the naming of log file
archives that are generated by the GetConnectorLogs servlet.
* Enhanced logging support for the testing environment by adding a
connector-manager/testdata/config/logging.properties file.
* Fixed the Connector Manager web application web.xml file to more
closely follow its DTD.
* Fixed a Daylight Saving Time bug in one of the tests. This issue
affected the test only, not the production Connector Manager.
Release 2.4.4, 05 February 2010
===============================
Introduction
------------
This is a patch release that fixes a few small problems discovered in the
previous release. Users of the previous release are encouraged to upgrade
immediately.
Summary of Changes
------------------
* Issue 209: Changed the default feed time zone from UTC to the
local time zone. Added a new configuration property, feed.timezone,
that may be set in applicationContext.properties if the local time
zone should not be used.
The 'feed.timezone' property defines the default time zone used
for Date metadata values for Documents. A null or empty string
indicates that the local time zone of the machine running the
Connector Manager should be used. Standard TimeZone identifiers
may be specified. For example:
feed.timezone=America/Los_Angeles
If a standard TimeZone identifier is unavailable, then a custom
TimeZone identifier can be constructed as +/-hours[minutes] offset
from GMT. For example:
feed.timezone=GMT+10 # GMT + 10 hours
feed.timezone=GMT+0630 # GMT + 6 hours, 30 minutes
feed.timezone=GMT-0800 # GMT - 8 hours, 0 minutes
This modification has compatibility implications when upgrading;
refer to the Version Compatibility section, below.
* Issue 211: Moved common Connector Manager configuration properties
from applicationContext.xml to applicationContext.properties.
This makes it much easier to upgrade to a newer version of the
Connector Manager, while preserving properties that had been
customized by the administrator.
This modification has compatibility implications when upgrading;
refer to the Version Compatibility section, below.
* Issue 212: Fixes "IOException: Attempted read on closed stream."
exceptions that might be generated if a Connector uses the
Apache Commons AutoCloseInputStream to provide document content
to the Connector Manager. Currently, only the SharePoint Connector
seems to have been affected by this problem.
Version Compatibility
---------------------
The time zone change has the most dramatic impact when upgrading.
Previous versions of Connector Manager would convert all date/time
values provided by the Connector to UTC dates when supplying the date
to the GSA. This had undesired consequences for users performing
date-range searches, where expected search results may have been
discarded as their adjusted date-stamps may have pushed the document
into the previous or next day. Consider this: a document modified
at 8:00PM PST today in California will have a calendar date of
tomorrow when adjusted to UTC time, so if you search for documents
modified today, it won't be found.
The Issue 209 change assumes date values supplied to the GSA are
local time, unless otherwise specified. This allows date-range
queries to function as expected when the Document Repository,
the Connector Manager, and the Search Appliance are in the same
time zone (or near enough time zones that 'normal working hours'
significantly overlap).
Upgrading an existing deployment to this new Connector Manager will
result in all newly indexed content feeding date values in the
local time zone, whereas all previously indexed content will have
UTC date values. The connector administrator may choose to handle
this inconsistency in one of several ways:
1) Do nothing. This may be desirable if date-range searches
for older materials need not be accurate to the day; or if