|
2014-01-31
§
|
| 14:45 |
<ottomata> |
analytics1022 back up with higher nofile ulimit, now handling all kafka traffic, analytics1021 wont' boot |
[production] |
| 14:10 |
<ottomata> |
upping noflie open file limit on analytics1021 and analytics1022, rebooting |
[production] |
| 06:59 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool db1052, warm up' |
[production] |
| 06:51 |
<springle> |
synchronized wmf-config/db-eqiad.php 'db1020 full steam. depool db1011 for schema changes' |
[production] |
| 05:03 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool db1052 for schema changes' |
[production] |
| 04:57 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool db1020, warm up' |
[production] |
| 03:41 |
<tstarling> |
finished scap: mostly no-op, did a reset to revert the wikidata changes (duration: 05m 00s) |
[production] |
| 03:37 |
<tstarling> |
started scap: mostly no-op, did a reset to revert the wikidata changes |
[production] |
| 02:07 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at 2014-01-31 02:07:39+00:00 |
[production] |
| 02:02 |
<LocalisationUpdate> |
completed (1.23wmf12) at 2014-01-31 02:02:37+00:00 |
[production] |
| 02:02 |
<LocalisationUpdate> |
completed (1.23wmf11) at 2014-01-31 02:01:50+00:00 |
[production] |
| 01:25 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool db1050 for schema changes' |
[production] |
| 00:19 |
<springle> |
xtrabackup clone db1042 to db1020 |
[production] |
| 00:19 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at 2014-01-31 00:19:28+00:00 |
[production] |
| 00:10 |
<LocalisationUpdate> |
completed (1.23wmf12) at 2014-01-31 00:10:17+00:00 |
[production] |
|
2014-01-30
§
|
| 23:40 |
<LocalisationUpdate> |
completed (1.23wmf11) at 2014-01-30 23:40:27+00:00 |
[production] |
| 23:14 |
<demon> |
synchronized wmf-config/Wikibase.php 'I50dfdc42: Enable quantity values' |
[production] |
| 22:59 |
<demon> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikidatawiki to wmf12 |
[production] |
| 22:36 |
<RobH> |
powercycling mw1061, system frozen |
[production] |
| 22:35 |
<RobH> |
rebooting frozen systems mw1057, mw1058, mw1060 |
[production] |
| 22:22 |
<RobH> |
powercycleing mw1039 as its also crashed |
[production] |
| 22:19 |
<RobH> |
mw1033 crashed, powercycling |
[production] |
| 22:14 |
<RobH> |
mw1036 crashed, unresponsive to console or ssh, rebooting |
[production] |
| 21:09 |
<reedy> |
synchronized php-1.23wmf12/extensions/Wikidata/ |
[production] |
| 20:38 |
<reedy> |
finished scap: Scap take 2 for 1.23wmf12 (duration: 18m 08s) |
[production] |
| 20:23 |
<reedy> |
started scap: Scap take 2 for 1.23wmf12 |
[production] |
| 20:16 |
<reedy> |
synchronized php-1.23wmf12/extensions/ContactPageFundraiser |
[production] |
| 20:04 |
<reedy> |
synchronized php-1.23wmf12/extensions/ContactPageFundraiser/ContactPage.php |
[production] |
| 19:55 |
<Reedy> |
Update indexes on wb_terms for testwikidatawiki https://gerrit.wikimedia.org/r/#/c/99660 |
[production] |
| 19:53 |
<Reedy> |
Changed wb_items_per_site.ips_row_id and wb_terms.term_row_id to BIGINT on testwikidatawiki |
[production] |
| 19:43 |
<reedy> |
synchronized php-1.23wmf12/includes/api/ApiQueryRevisions.php 'bug 60635' |
[production] |
| 19:36 |
<reedy> |
synchronized php-1.23wmf12/extensions/PdfHandler |
[production] |
| 19:31 |
<manybubbles> |
rebuilding search index on test2wiki went perfectly. proceeding with test, testwikidatawiki, and mediawikiwiki |
[production] |
| 19:29 |
<manybubbles> |
rebuilding search index for test2wiki and checking that everything is sane |
[production] |
| 19:25 |
<awight_> |
updated crm from 2516fe6fe95c1915c981fcad287fdd6119202344 to 600e09e35c59c3b8c127e5f3c3cf2df11db4f20e |
[production] |
| 19:20 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 19:20 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group0 wikis to 1.23wmf12 |
[production] |
| 19:19 |
<reedy> |
updated /a/common to {{Gerrit|Ie5c11239f}}: Update php symlink to php-1.23wmf11 |
[production] |
| 19:08 |
<Reedy> |
testwiki back to 1.23wmf11 even |
[production] |
| 19:06 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: all wikipedias to 1.23wmf11. testwiki back to 1.23wmf10 |
[production] |
| 19:02 |
<reedy> |
finished scap: testwiki to 1.23wmf12 and build l10n cache (duration: 18m 45s) |
[production] |
| 19:01 |
<awight_> |
crm updated from 9dc1887ce8391ca3a97dbb52d1d23454bbb21b03 to 2516fe6fe95c1915c981fcad287fdd6119202344 |
[production] |
| 18:57 |
<^d> |
jenkins: aborted another one of those pywiki jobs, was starting to back things up again. this job is broken methinks. |
[production] |
| 18:46 |
<reedy> |
started scap: testwiki to 1.23wmf12 and build l10n cache |
[production] |
| 18:42 |
<reedy> |
synchronized docroot and w |
[production] |
| 18:42 |
<reedy> |
synchronized php-1.23wmf12 'staging' |
[production] |
| 18:32 |
<^d> |
jenkins: killed job, queue seems to be trying to catch up now |
[production] |
| 18:31 |
<^d> |
jenkins backed up due to pywikibot-core-test job stuck again |
[production] |
| 18:26 |
<awight_> |
crm updated from 7c675028cf9a51d4b2d6c8f0d716c343225eb0db to 9dc1887ce8391ca3a97dbb52d1d23454bbb21b03 |
[production] |
| 17:51 |
<reedy> |
updated /a/common to {{Gerrit|Id0633e5a0}}: depool db1020 for schema changes |
[production] |