|
2014-11-22
§
|
| 23:02 |
<springle> |
upgrade db1020 trusty, xtrabackup clone db1046 to db1020 |
[production] |
| 21:21 |
<hashar> |
Jenkins: disconnected/reconnected gallium slave. All executors were being busy / deadlocked |
[production] |
| 20:29 |
<springle> |
db1046 m2-master threadpool lockup, restarted mysqld, investigating |
[production] |
| 19:39 |
<ori> |
Synchronized wmf-config/InitialiseSettings.php: Ifae6e0ab6: Clean up indents, comments, spacing in InitialiseSettings (duration: 00m 05s) |
[production] |
| 04:30 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Sat Nov 22 04:30:53 UTC 2014 (duration 30m 51s) |
[production] |
| 02:33 |
<LocalisationUpdate> |
completed (1.25wmf9) at 2014-11-22 02:33:36+00:00 |
[production] |
| 02:33 |
<l10nupdate> |
Synchronized php-1.25wmf9/cache/l10n: (no message) (duration: 00m 01s) |
[production] |
| 02:20 |
<LocalisationUpdate> |
completed (1.25wmf8) at 2014-11-22 02:20:36+00:00 |
[production] |
| 02:20 |
<l10nupdate> |
Synchronized php-1.25wmf8/cache/l10n: (no message) (duration: 00m 01s) |
[production] |
| 00:44 |
<hoo> |
Disabled login for dewiki accounts "W" and "H" |
[production] |
|
2014-11-21
§
|
| 23:33 |
<K4-713> |
Updated payments to 374480152a40d1b |
[production] |
| 23:28 |
<hoo> |
Disabled login for dewiki account "@" |
[production] |
| 22:54 |
<hoo> |
Disabled login for dewiki account "C" |
[production] |
| 22:17 |
<ejegg> |
Synchronized php-1.25wmf8/extensions/CentralNotice/: (no message) (duration: 00m 05s) |
[production] |
| 21:45 |
<anomie> |
Synchronized php-1.25wmf8/extensions/SecurePoll/: Backport another SecurePoll bug fix (duration: 00m 06s) |
[production] |
| 21:44 |
<anomie> |
Synchronized php-1.25wmf9/extensions/SecurePoll/: Backport another SecurePoll bug fix (duration: 00m 06s) |
[production] |
| 21:32 |
<anomie> |
Synchronized wmf-config/InitialiseSettings.php: Testing scap, no actual change (duration: 00m 05s) |
[production] |
| 21:25 |
<anomie> |
Synchronized wmf-config/InitialiseSettings.php: Testing scap, no actual change (duration: 00m 06s) |
[production] |
| 21:23 |
<anomie> |
Synchronized php-1.25wmf8/extensions/SecurePoll/: Backport SecurePoll bug fixes (duration: 00m 05s) |
[production] |
| 21:22 |
<anomie> |
Synchronized php-1.25wmf8/extensions/SecurePoll/: Backport SecurePoll bug fixes (duration: 00m 01s) |
[production] |
| 21:18 |
<ori> |
Synchronized php-1.25wmf9/extensions/SecurePoll: Backport SecurePoll bug fixes (duration: 00m 06s) |
[production] |
| 21:14 |
<anomie> |
Synchronized php-1.25wmf9/extensions/SecurePoll/: Backport SecurePoll bug fixes (duration: 00m 01s) |
[production] |
| 21:10 |
<ejegg> |
Synchronized php-1.25wmf9/extensions/CentralNotice/: (no message) (duration: 00m 07s) |
[production] |
| 20:31 |
<aaron> |
Synchronized wmf-config/InitialiseSettings.php: Removed duplicated BounceHandler log entry (duration: 00m 05s) |
[production] |
| 18:16 |
<chasemp> |
rebooting zirconium because bugzilla |
[production] |
| 17:26 |
<hoo> |
Disabled login for dewiki account "K" |
[production] |
| 17:19 |
<_joe_> |
apache hard restart on strontium |
[production] |
| 17:06 |
<bd808> |
deleted salt keys for deleted instances: i-00000289, i-0000028a, i-0000028b, i-0000028e, i-000002b7, i-000006ad |
[releng] |
| 16:59 |
<_joe_> |
restarted hhvm on mw1025, TC cache exhausted |
[production] |
| 16:43 |
<_joe_> |
pooled mw1228-9 |
[production] |
| 16:33 |
<_joe_> |
pooling mw1236 (HHVM) into the main apache pool |
[production] |
| 16:23 |
<_joe_> |
repooling mw1232-3 |
[production] |
| 16:17 |
<godog> |
upload carbonate 0.2.2-1 to trusty-wikimedia |
[production] |
| 15:57 |
<hashar> |
fixed puppet cert on deployment-restbase01 |
[releng] |
| 15:56 |
<_joe_> |
repooling mw1231 |
[production] |
| 15:52 |
<_joe_> |
repooling mw1230 |
[production] |
| 15:50 |
<hashar> |
deployment-sca01 regenerating puppet CA for deployment-sca01 |
[releng] |
| 15:45 |
<_joe_> |
repooling mw1227 |
[production] |
| 15:34 |
<hashar> |
Renerated puppet master certificate on deployment-salt. It needs to be named deployment-salt.eqiad.wmflabs not i-0000015c.eqiad.wmflabs. Puppet agent works on deployment-salt now. |
[releng] |
| 15:19 |
<hashar> |
I have revoked the deployment-salt certificates. All puppet agent are thus broken! |
[releng] |
| 15:01 |
<hashar> |
deployment-salt cleaning certs with puppet cert clean |
[releng] |
| 14:57 |
<hoo> |
Synchronized wmf-config/InitialiseSettings.php: Add 'BounceHandler' to wgDebugLogGroups (duration: 00m 05s) |
[production] |
| 14:52 |
<hashar> |
manually switching restbase01 puppet master from virt1000 to deployment-salt.eqiad.wmflabs |
[releng] |
| 14:50 |
<hashar> |
deployment-restbase01 has some puppet error: Error 400 on SERVER: Must provide non empty value. on node i-00000727.eqiad.wmflabs . That is due to puppet pickle() function being given an empty variable |
[releng] |
| 14:47 |
<cmjohnson> |
mw1230-1233 down --reinstalling |
[production] |
| 14:34 |
<hoo> |
Synchronized wmf-config/CommonSettings.php: Also whitelist IPv6 ips for bouncehandler (duration: 00m 08s) |
[production] |
| 13:48 |
<_joe_> |
depooled mw1227 |
[production] |
| 13:08 |
<springle> |
fresh dump db1046 to db2011 |
[production] |
| 11:39 |
<mark> |
Synchronized wmf-config/InitialiseSettings.php: add openfashion.momu.be to wgCopyUploadsDomains (duration: 00m 06s) |
[production] |
| 11:05 |
<_joe_> |
pooled mw1234,mw1235 in the api pool |
[production] |