CalculateImpliedTimescales.py: Abnormal termination of BFGS likelihood maximization. Error code 2

[davidshorthouse]

CalculateImpliedTimscales.py gives errors when running on Tutorial data. Digging into the code, the error stems from MSMLib.py class Reversible_MLE_Estimator. The logger at line 1238 in MSMLib.py throws the error.

Looking into the scipy.optimize.fmin_l_bfgs_b, the error code 2 is associated with an error in the task: ABNORMAL_TERMINATION_IN_LNSRCH.

Is this a bug, or can these be safely ignored without affecting the output? If it can be ignored, what values can I use to check this - The number of times BFGS is called? The difference between the initial and final log likelihoods?

Thanks!

A log of the output is below:

msmb CalculateImpliedTimescales -l 1,25 -i 1 -o Data/ImpliedTimescales.dat

{'assignments': 'Data/Assignments.h5',
'eigvals': 10,
'interval': 1,
'lagtime': '1,25',
'notrim': False,
'output': 'Data/ImpliedTimescales.dat',
'procs': 1,
'quiet': False,
'symmetrize': 'MLE'}
11:18:06 - Getting 10 eigenvalues (timescales) for each lagtime...
11:18:06 - Building MSMs at the following lag times: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25]
11:18:07 - Calculating implied timescales at lagtime 1
11:18:07 - Selected component 0 with population 1.000000
11:18:07 - BFGS likelihood maximization terminated after 179 function calls. Initial and final log likelihoods: -139753.326799, -139753.280195.
11:18:07 - Calculating implied timescales at lagtime 2
11:18:07 - Selected component 0 with population 1.000000
11:18:08 - BFGS likelihood maximization terminated after 290 function calls. Initial and final log likelihoods: -151179.602729, -151179.513937.
11:18:08 - Abnormal termination of BFGS likelihood maximization. Error code 2
11:18:08 - Calculating implied timescales at lagtime 3
11:18:08 - Selected component 0 with population 1.000000
11:18:09 - BFGS likelihood maximization terminated after 208 function calls. Initial and final log likelihoods: -156015.822473, -156015.714189.
11:18:09 - Calculating implied timescales at lagtime 4
11:18:09 - Selected component 0 with population 1.000000
11:18:09 - BFGS likelihood maximization terminated after 150 function calls. Initial and final log likelihoods: -158668.090860, -158667.975200.
11:18:09 - Calculating implied timescales at lagtime 5
11:18:09 - Selected component 0 with population 1.000000
11:18:10 - BFGS likelihood maximization terminated after 159 function calls. Initial and final log likelihoods: -160284.118258, -160283.984699.
11:18:10 - Calculating implied timescales at lagtime 6
11:18:10 - Selected component 0 with population 1.000000
11:18:10 - BFGS likelihood maximization terminated after 191 function calls. Initial and final log likelihoods: -161379.312536, -161379.210149.
11:18:10 - Abnormal termination of BFGS likelihood maximization. Error code 2
11:18:10 - Calculating implied timescales at lagtime 7
11:18:10 - Selected component 0 with population 1.000000
11:18:11 - BFGS likelihood maximization terminated after 214 function calls. Initial and final log likelihoods: -162231.657654, -162231.542798.
11:18:11 - Abnormal termination of BFGS likelihood maximization. Error code 2
11:18:11 - Calculating implied timescales at lagtime 8
11:18:11 - Selected component 0 with population 1.000000
11:18:12 - BFGS likelihood maximization terminated after 225 function calls. Initial and final log likelihoods: -162866.406582, -162866.266906.
11:18:12 - Abnormal termination of BFGS likelihood maximization. Error code 2
11:18:12 - Calculating implied timescales at lagtime 9
11:18:12 - Selected component 0 with population 1.000000
11:18:12 - BFGS likelihood maximization terminated after 168 function calls. Initial and final log likelihoods: -163313.274829, -163313.109101.
11:18:12 - Calculating implied timescales at lagtime 10
11:18:12 - Selected component 0 with population 1.000000
11:18:13 - BFGS likelihood maximization terminated after 142 function calls. Initial and final log likelihoods: -163667.316070, -163667.133324.
11:18:13 - Calculating implied timescales at lagtime 11
11:18:13 - Selected component 0 with population 1.000000
11:18:13 - BFGS likelihood maximization terminated after 159 function calls. Initial and final log likelihoods: -163864.187442, -163863.978430.
11:18:13 - Calculating implied timescales at lagtime 12
11:18:13 - Selected component 0 with population 1.000000
11:18:14 - BFGS likelihood maximization terminated after 145 function calls. Initial and final log likelihoods: -164045.527258, -164045.308630.
11:18:14 - Calculating implied timescales at lagtime 13
11:18:14 - Selected component 0 with population 1.000000
11:18:14 - BFGS likelihood maximization terminated after 227 function calls. Initial and final log likelihoods: -164161.629676, -164161.388940.
11:18:14 - Abnormal termination of BFGS likelihood maximization. Error code 2
11:18:14 - Calculating implied timescales at lagtime 14
11:18:14 - Selected component 0 with population 1.000000
11:18:15 - BFGS likelihood maximization terminated after 141 function calls. Initial and final log likelihoods: -164182.820106, -164182.553320.
11:18:15 - Calculating implied timescales at lagtime 15
11:18:15 - Selected component 0 with population 1.000000
11:18:15 - BFGS likelihood maximization terminated after 201 function calls. Initial and final log likelihoods: -164155.829783, -164155.516090.
11:18:15 - Abnormal termination of BFGS likelihood maximization. Error code 2
11:18:16 - Calculating implied timescales at lagtime 16
11:18:16 - Selected component 0 with population 1.000000
11:18:16 - BFGS likelihood maximization terminated after 206 function calls. Initial and final log likelihoods: -164079.151689, -164078.802164.
11:18:16 - Abnormal termination of BFGS likelihood maximization. Error code 2
11:18:16 - Calculating implied timescales at lagtime 17
11:18:16 - Selected component 0 with population 1.000000
11:18:17 - BFGS likelihood maximization terminated after 149 function calls. Initial and final log likelihoods: -163928.442116, -163928.041659.
11:18:17 - Calculating implied timescales at lagtime 18
11:18:17 - Selected component 0 with population 1.000000
11:18:17 - BFGS likelihood maximization terminated after 146 function calls. Initial and final log likelihoods: -163770.627190, -163770.173608.
11:18:17 - Calculating implied timescales at lagtime 19
11:18:17 - Selected component 0 with population 1.000000
11:18:18 - BFGS likelihood maximization terminated after 198 function calls. Initial and final log likelihoods: -163601.944893, -163601.485538.
11:18:18 - Calculating implied timescales at lagtime 20
11:18:18 - Selected component 0 with population 1.000000
11:18:18 - BFGS likelihood maximization terminated after 195 function calls. Initial and final log likelihoods: -163386.614756, -163386.108368.
11:18:18 - Abnormal termination of BFGS likelihood maximization. Error code 2
11:18:18 - Calculating implied timescales at lagtime 21
11:18:18 - Selected component 0 with population 1.000000
11:18:19 - BFGS likelihood maximization terminated after 187 function calls. Initial and final log likelihoods: -163209.905438, -163209.364213.
11:18:19 - Abnormal termination of BFGS likelihood maximization. Error code 2
11:18:19 - Calculating implied timescales at lagtime 22
11:18:19 - Selected component 0 with population 1.000000
11:18:20 - BFGS likelihood maximization terminated after 146 function calls. Initial and final log likelihoods: -163004.085613, -163003.454112.
11:18:20 - Calculating implied timescales at lagtime 23
11:18:20 - Selected component 0 with population 1.000000
11:18:20 - BFGS likelihood maximization terminated after 194 function calls. Initial and final log likelihoods: -162746.629915, -162745.948430.
11:18:20 - Abnormal termination of BFGS likelihood maximization. Error code 2
11:18:20 - Calculating implied timescales at lagtime 24
11:18:20 - Selected component 0 with population 1.000000
11:18:21 - BFGS likelihood maximization terminated after 147 function calls. Initial and final log likelihoods: -162472.918951, -162472.180937.
11:18:21 - Calculating implied timescales at lagtime 25
11:18:21 - Selected component 0 with population 1.000000
11:18:21 - BFGS likelihood maximization terminated after 204 function calls. Initial and final log likelihoods: -162201.056782, -162200.278590.
11:18:21 - Abnormal termination of BFGS likelihood maximization. Error code 2
11:18:21 - Saved output to Data/ImpliedTimescales.dat

[kyleabeauchamp]

I think this is somewhat normal.
On Jun 12, 2014 6:35 AM, "monstrouspaddling" [email protected]
wrote:

CalculateImpliedTimscales.py gives errors when running on Tutorial data.
Digging into the code, the error stems from MSMLib.py class
Reversible_MLE_Estimator. The logger at line 1238 in MSMLib.py throws the
error.

Looking into the scipy.optimize.fmin_l_bfgs_b, the error code 2 is
associated with an error in the task: ABNORMAL_TERMINATION_IN_LNSRCH.

Is this a bug, or can these be safely ignored without affecting the
output? If it can be ignored, what values can I use to check this - The
number of times BFGS is called? The difference between the initial and
final log likelihoods?

Thanks!

A log of the output is below:

msmb CalculateImpliedTimescales -l 1,25 -i 1 -o Data/ImpliedTimescales.dat

{'assignments': 'Data/Assignments.h5',
'eigvals': 10,
'interval': 1,
'lagtime': '1,25',
'notrim': False,
'output': 'Data/ImpliedTimescales.dat',
'procs': 1,
'quiet': False,
'symmetrize': 'MLE'}
11:18:06 - Getting 10 eigenvalues (timescales) for each lagtime...
11:18:06 - Building MSMs at the following lag times: [1, 2, 3, 4, 5, 6, 7,
8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25]
11:18:07 - Calculating implied timescales at lagtime 1
11:18:07 - Selected component 0 with population 1.000000
11:18:07 - BFGS likelihood maximization terminated after 179 function
calls. Initial and final log likelihoods: -139753.326799, -139753.280195.
11:18:07 - Calculating implied timescales at lagtime 2
11:18:07 - Selected component 0 with population 1.000000
11:18:08 - BFGS likelihood maximization terminated after 290 function
calls. Initial and final log likelihoods: -151179.602729, -151179.513937.
11:18:08 - Abnormal termination of BFGS likelihood maximization. Error
code 2
11:18:08 - Calculating implied timescales at lagtime 3
11:18:08 - Selected component 0 with population 1.000000
11:18:09 - BFGS likelihood maximization terminated after 208 function
calls. Initial and final log likelihoods: -156015.822473, -156015.714189.
11:18:09 - Calculating implied timescales at lagtime 4
11:18:09 - Selected component 0 with population 1.000000
11:18:09 - BFGS likelihood maximization terminated after 150 function
calls. Initial and final log likelihoods: -158668.090860, -158667.975200.
11:18:09 - Calculating implied timescales at lagtime 5
11:18:09 - Selected component 0 with population 1.000000
11:18:10 - BFGS likelihood maximization terminated after 159 function
calls. Initial and final log likelihoods: -160284.118258, -160283.984699.
11:18:10 - Calculating implied timescales at lagtime 6
11:18:10 - Selected component 0 with population 1.000000
11:18:10 - BFGS likelihood maximization terminated after 191 function
calls. Initial and final log likelihoods: -161379.312536, -161379.210149.
11:18:10 - Abnormal termination of BFGS likelihood maximization. Error
code 2
11:18:10 - Calculating implied timescales at lagtime 7
11:18:10 - Selected component 0 with population 1.000000
11:18:11 - BFGS likelihood maximization terminated after 214 function
calls. Initial and final log likelihoods: -162231.657654, -162231.542798.
11:18:11 - Abnormal termination of BFGS likelihood maximization. Error
code 2
11:18:11 - Calculating implied timescales at lagtime 8
11:18:11 - Selected component 0 with population 1.000000
11:18:12 - BFGS likelihood maximization terminated after 225 function
calls. Initial and final log likelihoods: -162866.406582, -162866.266906.
11:18:12 - Abnormal termination of BFGS likelihood maximization. Error
code 2
11:18:12 - Calculating implied timescales at lagtime 9
11:18:12 - Selected component 0 with population 1.000000
11:18:12 - BFGS likelihood maximization terminated after 168 function
calls. Initial and final log likelihoods: -163313.274829, -163313.109101.
11:18:12 - Calculating implied timescales at lagtime 10
11:18:12 - Selected component 0 with population 1.000000
11:18:13 - BFGS likelihood maximization terminated after 142 function
calls. Initial and final log likelihoods: -163667.316070, -163667.133324.
11:18:13 - Calculating implied timescales at lagtime 11
11:18:13 - Selected component 0 with population 1.000000
11:18:13 - BFGS likelihood maximization terminated after 159 function
calls. Initial and final log likelihoods: -163864.187442, -163863.978430.
11:18:13 - Calculating implied timescales at lagtime 12
11:18:13 - Selected component 0 with population 1.000000
11:18:14 - BFGS likelihood maximization terminated after 145 function
calls. Initial and final log likelihoods: -164045.527258, -164045.308630.
11:18:14 - Calculating implied timescales at lagtime 13
11:18:14 - Selected component 0 with population 1.000000
11:18:14 - BFGS likelihood maximization terminated after 227 function
calls. Initial and final log likelihoods: -164161.629676, -164161.388940.
11:18:14 - Abnormal termination of BFGS likelihood maximization. Error
code 2
11:18:14 - Calculating implied timescales at lagtime 14
11:18:14 - Selected component 0 with population 1.000000
11:18:15 - BFGS likelihood maximization terminated after 141 function
calls. Initial and final log likelihoods: -164182.820106, -164182.553320.
11:18:15 - Calculating implied timescales at lagtime 15
11:18:15 - Selected component 0 with population 1.000000
11:18:15 - BFGS likelihood maximization terminated after 201 function
calls. Initial and final log likelihoods: -164155.829783, -164155.516090.
11:18:15 - Abnormal termination of BFGS likelihood maximization. Error
code 2
11:18:16 - Calculating implied timescales at lagtime 16
11:18:16 - Selected component 0 with population 1.000000
11:18:16 - BFGS likelihood maximization terminated after 206 function
calls. Initial and final log likelihoods: -164079.151689, -164078.802164.
11:18:16 - Abnormal termination of BFGS likelihood maximization. Error
code 2
11:18:16 - Calculating implied timescales at lagtime 17
11:18:16 - Selected component 0 with population 1.000000
11:18:17 - BFGS likelihood maximization terminated after 149 function
calls. Initial and final log likelihoods: -163928.442116, -163928.041659.
11:18:17 - Calculating implied timescales at lagtime 18
11:18:17 - Selected component 0 with population 1.000000
11:18:17 - BFGS likelihood maximization terminated after 146 function
calls. Initial and final log likelihoods: -163770.627190, -163770.173608.
11:18:17 - Calculating implied timescales at lagtime 19
11:18:17 - Selected component 0 with population 1.000000
11:18:18 - BFGS likelihood maximization terminated after 198 function
calls. Initial and final log likelihoods: -163601.944893, -163601.485538.
11:18:18 - Calculating implied timescales at lagtime 20
11:18:18 - Selected component 0 with population 1.000000
11:18:18 - BFGS likelihood maximization terminated after 195 function
calls. Initial and final log likelihoods: -163386.614756, -163386.108368.
11:18:18 - Abnormal termination of BFGS likelihood maximization. Error
code 2
11:18:18 - Calculating implied timescales at lagtime 21
11:18:18 - Selected component 0 with population 1.000000
11:18:19 - BFGS likelihood maximization terminated after 187 function
calls. Initial and final log likelihoods: -163209.905438, -163209.364213.
11:18:19 - Abnormal termination of BFGS likelihood maximization. Error
code 2
11:18:19 - Calculating implied timescales at lagtime 22
11:18:19 - Selected component 0 with population 1.000000
11:18:20 - BFGS likelihood maximization terminated after 146 function
calls. Initial and final log likelihoods: -163004.085613, -163003.454112.
11:18:20 - Calculating implied timescales at lagtime 23
11:18:20 - Selected component 0 with population 1.000000
11:18:20 - BFGS likelihood maximization terminated after 194 function
calls. Initial and final log likelihoods: -162746.629915, -162745.948430.
11:18:20 - Abnormal termination of BFGS likelihood maximization. Error
code 2
11:18:20 - Calculating implied timescales at lagtime 24
11:18:20 - Selected component 0 with population 1.000000
11:18:21 - BFGS likelihood maximization terminated after 147 function
calls. Initial and final log likelihoods: -162472.918951, -162472.180937.
11:18:21 - Calculating implied timescales at lagtime 25
11:18:21 - Selected component 0 with population 1.000000
11:18:21 - BFGS likelihood maximization terminated after 204 function
calls. Initial and final log likelihoods: -162201.056782, -162200.278590.
11:18:21 - Abnormal termination of BFGS likelihood maximization. Error
code 2
11:18:21 - Saved output to Data/ImpliedTimescales.dat

—
Reply to this email directly or view it on GitHub
#425.

[davidshorthouse]

Alright - thanks for the response!