Page MenuHomePhabricator
Create Task
Maniphest T104

CCC: Rcworker.py time data mismatch value error
Closed, ResolvedPublicBUG
Actions

Description

goed.jpg :Not corrupt. Stored
2020-02-08 00:50:15,448 __main__    : INFO File:Agnes Tilney.jpg
Traceback (most recent call last):
  File "rcworker.py", line 222, in <module>
    main()
  File "rcworker.py", line 215, in main
    run_worker()
  File "rcworker.py", line 109, in run_worker
    revision = change.getRevision(file_page)
  File "/home/ccc/Commons-image-corruption-detector/Image.py", line 24, in getRevision
    revision = file_page.get_file_history()[pywikibot.Timestamp.fromtimestampformat(self.log_timestamp)]
  File "/usr/local/lib/python3.8/site-packages/pywikibot/__init__.py", line 210, in fromtimestampformat
    return cls.strptime(ts, cls.mediawikiTSFormat)
  File "/usr/local/lib/python3.8/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/local/lib/python3.8/_strptime.py", line 349, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data '2010-11-14T16:09:58Z' does not match format '%Y%m%d%H%M%S'

Revisions and Commits

Restricted Diffusion Commit
Restricted Diffusion Commit
Restricted Diffusion Commit
Restricted Diffusion Commit

Related Objects

Event Timeline

TheSandDoctor triaged this task as Normal priority.Feb 7 2020, 4:55 PM
TheSandDoctor created this task.
TheSandDoctor created this object with visibility "Public (No Login Required)".
TheSandDoctor added a comment.Feb 7 2020, 5:16 PM

Initial tests do not appear able to reproduce this locally.

TheSandDoctor added a commit: Restricted Diffusion Commit.Feb 7 2020, 5:21 PM
TheSandDoctor updated the task description. (Show Details)Feb 7 2020, 5:27 PM

Cannot reproduce locally nor on server side.

TheSandDoctor added a comment.Feb 7 2020, 5:32 PM

Just happened again at random.

2020-02-08 01:32:01,153 __main__    : INFO File:AIIPlogo3d.png
Traceback (most recent call last):
  File "rcworker.py", line 216, in <module>
    main()
  File "rcworker.py", line 209, in main
    run_worker()
  File "rcworker.py", line 105, in run_worker
    revision = change.getRevision(file_page)
  File "/home/ccc/Commons-image-corruption-detector/Image.py", line 24, in getRevision
    revision = file_page.get_file_history()[pywikibot.Timestamp.fromtimestampformat(self.log_timestamp)]
  File "/usr/local/lib/python3.8/site-packages/pywikibot/__init__.py", line 210, in fromtimestampformat
    return cls.strptime(ts, cls.mediawikiTSFormat)
  File "/usr/local/lib/python3.8/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/local/lib/python3.8/_strptime.py", line 349, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data '2020-01-17T13:54:19Z' does not match format '%Y%m%d%H%M%S'
TheSandDoctor added a commit: Restricted Diffusion Commit.Feb 7 2020, 7:55 PM
TheSandDoctor added a commit: Restricted Diffusion Commit.Feb 7 2020, 8:23 PM
TheSandDoctor added a comment.Feb 8 2020, 12:33 AM
2020-02-08 07:54:06,423 __main__    : INFO File:Yackety yack (1901) - DPLA - 30e4078df08e452cbcded5fd72443251 (page 246).jpg :Not corrupt. Stored
2020-02-08 07:54:06,426 __main__    : INFO File:Regina Doherty 2015.jpg
2020-02-08 07:54:06,666 Image       : WARNING KeyError1 has occurred
Traceback (most recent call last):
  File "/home/ccc/Commons-image-corruption-detector/Image.py", line 29, in getRevision
    revision = file_page.get_file_history()[pywikibot.Timestamp.fromtimestampformat(self.log_timestamp)]
KeyError: Timestamp(2020, 2, 8, 7, 44, 20)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "rcworker.py", line 216, in <module>
    main()
  File "rcworker.py", line 209, in main
    run_worker()
  File "rcworker.py", line 105, in run_worker
    revision = change.getRevision(file_page)
  File "/home/ccc/Commons-image-corruption-detector/Image.py", line 34, in getRevision
    revision = file_page.get_file_history()[pywikibot.Timestamp.fromISOformat(self.log_timestamp)]
  File "/usr/local/lib/python3.8/site-packages/pywikibot/__init__.py", line 199, in fromISOformat
    return cls.strptime(ts, cls._ISO8601Format(sep))
  File "/usr/local/lib/python3.8/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/local/lib/python3.8/_strptime.py", line 349, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data '20200208074420' does not match format '%Y-%m-%dT%H:%M:%SZ'
TheSandDoctor added a comment.Feb 11 2020, 8:59 AM

Should catch error, log locally, and continue to the next image. Last night it stalled out at some point due to this, after having run for over a day without issues.

TheSandDoctor added a commit: Restricted Diffusion Commit.Feb 12 2020, 8:26 AM
TheSandDoctor added a comment.Feb 12 2020, 8:26 AM

Another crash at some point last night. One commit pushed/pulled, but I am too busy to continue at the moment. Will work more on this tonight most likely.

TheSandDoctor added a comment.Feb 12 2020, 8:27 AM

Both rcwatcher and rcworker have been restarted with the latest patch.

TheSandDoctor changed the subtype of this task from "Task" to "Bug".Feb 12 2020, 8:28 AM
TheSandDoctor mentioned this in T123: CCC: Investigate File:Tarnow Park Strzelecki wiewiorka 5.jpg corruption.Feb 13 2020, 11:47 AM
TheSandDoctor closed this task as Resolved.Apr 8 2020, 5:03 PM

Rcworker has since been rebuilt and this has not re-emerged since. Closing.