|
2014-12-18
§
|
| 18:25 |
<demon> |
Synchronized wmf-config/InitialiseSettings.php: logging for IP argument bugs (duration: 00m 05s) |
[production] |
| 18:09 |
<bblack> |
analytics vlan ACLs updated in eqiad |
[production] |
| 17:59 |
<demon> |
Synchronized wmf-config/: Disable lsearchd almost everywhere (duration: 00m 07s) |
[production] |
| 17:42 |
<demon> |
Synchronized wmf-config/: Remove cirrus-as-alternate settings (duration: 00m 06s) |
[production] |
| 17:22 |
<bblack> |
switched to new unified cert on all nginx terminators via config reload |
[production] |
| 17:18 |
<godog> |
enabling md write intent bitmap temporarily on virt1009 |
[production] |
| 17:08 |
<hashar> |
Made mediawiki-phpunit-hhvm Jenkins job voting. We now enforce HHVM compliance for mediawiki/core |
[production] |
| 16:37 |
<hashar> |
gallium deleting obsoletes jobs: rm -fR /srv/ssd/jenkins-slave/workspace/*-testextension . They are now suffixed with -zend and -hhvm |
[production] |
| 16:35 |
<godog> |
deleting jenkins workspaces on lanthanum older than 30d |
[production] |
| 16:32 |
<hashar> |
lanthanum deleting obsoletes jobs: rm -fR /srv/ssd/jenkins-slave/workspace/*-testextension . They are now suffixed with -zend and -hhvm |
[production] |
| 16:24 |
<demon> |
Synchronized wmf-config/StartProfiler.php: disable normal eqiad profiling (duration: 00m 06s) |
[production] |
| 16:13 |
<manybubbles> |
Synchronized php-1.25wmf12/extensions/MultimediaViewer/: SWAT backport last-modified performance logging for mediaviewer (duration: 00m 05s) |
[production] |
| 16:06 |
<manybubbles> |
Synchronized wmf-config/InitialiseSettings.php: SWAT disable thumbnail caching (duration: 00m 05s) |
[production] |
| 16:06 |
<manybubbles> |
Synchronized wmf-config/InitialiseSettings-labs.php: SWAT disable thumbnail caching (duration: 00m 05s) |
[production] |
| 15:43 |
<_joe_> |
rebooted mw1191 |
[production] |
| 15:07 |
<demon> |
Synchronized wmf-config/StartProfiler.php: disable xhprof-backed flame graphs for now (duration: 00m 05s) |
[production] |
| 14:57 |
<_joe_> |
rebooting mw1257 |
[production] |
| 14:44 |
<_joe_> |
restarting hhvm on mw1191 |
[production] |
| 14:01 |
<qchris> |
EventLogging: deployed 937d804 & restarted EventLogging |
[production] |
| 13:54 |
<akosiaris> |
purged unpuppetized rrdcached from hafnium. It was segfaulting when started via the init script, which led to the package being unconfigured which led to dpkg alerts on icinga |
[production] |
| 13:42 |
<_joe|lunch> |
restarting hhvm on a few servers |
[production] |
| 13:11 |
<_joe_> |
restarted hhvm on mw1242, stuck in getrusage() |
[production] |
| 13:03 |
<_joe_> |
restarted hhvm on mw1191, load at 200 |
[production] |
| 13:00 |
<paravoid> |
salt-cleaning up /etc/sudoers.d/50_* (old naming scheme) |
[production] |
| 12:04 |
<godog> |
upload carbon-c-relay 0.36+git20141218-1 to trusty-wikimedia |
[production] |
| 09:13 |
<hashar> |
enabled MediaWiki core 'structure' PHPUnit tests for all extensions. Will require folks to fix their incorrect AutoLoader and RessourceLoader entries. {{gerrit|180496}} {{bug|T78798}} |
[production] |
| 06:23 |
<_joe|justawake> |
restarted the puppetmaster on palladium |
[production] |
| 04:35 |
<springle> |
Synchronized wmf-config/db-eqiad.php: depool db1055 (duration: 04m 19s) |
[production] |
| 01:42 |
<ori> |
Synchronized php-1.25wmf12/resources/src/mediawiki.action/mediawiki.action.edit.stash.js: Ibb29a825c: mediawiki.action.edit.stash: set timeout to 4 seconds (duration: 00m 05s) |
[production] |
| 01:31 |
<awight> |
update crm from f1e558592ee98ff8fc84d19ff2c0435619e11242 to ae7b2381667dd65d68812c58f61e3ea66fa9fa6f |
[production] |
| 01:25 |
<springle> |
Synchronized wmf-config/db-eqiad.php: depool db1072 (duration: 00m 08s) |
[production] |
| 01:22 |
<springle> |
mw1191 restarted hhvm, apparently stuck in futex |
[production] |
| 00:38 |
<springle> |
Synchronized wmf-config/db-eqiad.php: repool db1066, warm up (duration: 00m 05s) |
[production] |
| 00:24 |
<hashar> |
Restarting Jenkins to remove a deadlock on deployment-bastion slave |
[production] |
| 00:19 |
<maxsem> |
Synchronized php-1.25wmf13/extensions/Flow/: https://gerrit.wikimedia.org/r/#/c/180677/ (duration: 00m 08s) |
[production] |
| 00:01 |
<maxsem> |
Synchronized wmf-config/: https://gerrit.wikimedia.org/r/#/c/180376/ - no-op for prod (duration: 00m 06s) |
[production] |
|
2014-12-17
§
|
| 23:34 |
<hashar> |
Restarted Jenkins and Zuul again to have a clean start while I am crashing to bed. |
[production] |
| 23:22 |
<demon> |
Synchronized wmf-config/StartProfiler.php: xhprof on all hhvm hosts in eqiad (duration: 00m 05s) |
[production] |
| 22:46 |
<hashar> |
restarting Jenkins |
[production] |
| 21:45 |
<hashar> |
killing Jenkins |
[production] |
| 21:41 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.25wmf13 |
[production] |
| 21:40 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.25wmf12 |
[production] |
| 21:38 |
<reedy> |
Finished scap: testwiki to 1.25wmf13 and build l10n cache (duration: 12m 26s) |
[production] |
| 21:25 |
<reedy> |
Started scap: testwiki to 1.25wmf13 and build l10n cache |
[production] |
| 21:24 |
<Reedy> |
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ for mw1152 |
[production] |
| 20:36 |
<hashar> |
Jenkins/Zuul had some deadlock. Disconnected/reconnected slaves but that did not fix it. Finally had to disconnect/reconnect thegearman client in Jenkins and it is processing again. |
[production] |
| 20:36 |
<reedy> |
Started scap: testwiki to 1.25wmf13 and build l10n cache |
[production] |
| 20:12 |
<hashar> |
Jenkins some slaves are no more properly registered. Unpooling / Repooling them |
[production] |
| 16:42 |
<demon> |
Synchronized php-1.25wmf12/extensions/TextExtracts/: (no message) (duration: 00m 05s) |
[production] |
| 16:27 |
<demon> |
Synchronized wmf-config/CirrusSearch-labs.php: for completeness (duration: 00m 05s) |
[production] |