|
2013-08-22
§
|
| 13:04 |
<paravoid> |
changed watchmouse alert and re-enabled |
[production] |
| 11:36 |
<paravoid> |
switching off alerts for watchmouse's DNS alert while investigating |
[production] |
| 11:03 |
<paravoid> |
disabling puppet on eeden for the next couple of days |
[production] |
| 02:38 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Aug 22 02:38:17 UTC 2013 |
[production] |
| 02:28 |
<LocalisationUpdate> |
completed (1.22wmf12) at Thu Aug 22 02:28:28 UTC 2013 |
[production] |
| 02:15 |
<LocalisationUpdate> |
completed (1.22wmf13) at Thu Aug 22 02:15:07 UTC 2013 |
[production] |
| 00:04 |
<catrope> |
synchronized php-1.22wmf13/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.ViewPageTarget.js 'touch' |
[production] |
| 00:04 |
<catrope> |
synchronized php-1.22wmf12/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.ViewPageTarget.js 'touch' |
[production] |
| 00:02 |
<catrope> |
synchronized php-1.22wmf13/extensions/VisualEditor 'Update VE to master' |
[production] |
| 00:01 |
<catrope> |
synchronized php-1.22wmf12/extensions/VisualEditor 'Update VE to master' |
[production] |
|
2013-08-21
§
|
| 22:31 |
<binasher> |
opened up wikiadmin grants on es1 shard to accommodate snapshot1004 |
[production] |
| 20:59 |
<mutante> |
cp1061,cp1062 have dpkg issue upgrading varnish packages |
[production] |
| 20:36 |
<mutante> |
installing package upgrades on iron |
[production] |
| 20:35 |
<mutante> |
re-enabled puppet runs on zirconium which was "admin down" |
[production] |
| 20:20 |
<mutante> |
dist-upgrading zirconium |
[production] |
| 18:41 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Disable Collection on testwikidatawiki. It's mucho brokenp' |
[production] |
| 18:38 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Enable Collection on testwikidatawiki' |
[production] |
| 18:29 |
<aaron> |
synchronized wmf-config/CommonSettings.php 'Enabled $wgMWOAuthSecureTokenTransfer' |
[production] |
| 16:36 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 16:27 |
<paravoid> |
DNS migration complete |
[production] |
| 16:21 |
<paravoid> |
switching ns2 to eeden |
[production] |
| 16:04 |
<reedy> |
synchronized docroot and w |
[production] |
| 15:51 |
<paravoid> |
switching ns0 to rubidium |
[production] |
| 15:46 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 15:46 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 15:36 |
<faidon> |
synchronized wmf-config/filebackend.php 'revert ceph promotion to master' |
[production] |
| 13:58 |
<paravoid> |
stopping ceph-osd 88 & 95 (ms-be1008), evidence of unexplainable high cpu usage |
[production] |
| 13:38 |
<paravoid> |
switching ns1 traffic to mexia (new authdns) |
[production] |
| 12:08 |
<Reedy> |
Running extensions/CentralAuth/maintenance/populateHomeDB.php in a screen on tin |
[production] |
| 11:02 |
<faidon> |
synchronized wmf-config/filebackend.php 'promote ceph to master' |
[production] |
| 08:01 |
<nikerabbit> |
synchronized php-1.22wmf13/extensions/UniversalLanguageSelector/ 'ULS to master' |
[production] |
| 02:46 |
<mutante> |
RT - create new system dashboard 'wikimedia default' that lists open ops-requests, quick ticket creation, reminders and bookmarked tickets |
[production] |
| 02:46 |
<mutante> |
RT - grant global right to see system dashboards to privileged users |
[production] |
| 02:35 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed Aug 21 02:35:19 UTC 2013 |
[production] |
| 02:28 |
<LocalisationUpdate> |
completed (1.22wmf12) at Wed Aug 21 02:28:33 UTC 2013 |
[production] |
| 02:15 |
<LocalisationUpdate> |
completed (1.22wmf13) at Wed Aug 21 02:15:12 UTC 2013 |
[production] |
| 01:03 |
<mwalker> |
synchronized php-1.22wmf12/extensions/CentralNotice/special 'Applying CentralNotice fix for bug 53032' |
[production] |
| 01:02 |
<mwalker> |
synchronized php-1.22wmf13/extensions/CentralNotice/special 'Applying CentralNotice fix for bug 53032' |
[production] |
| 00:58 |
<mutante> |
allow engineering and ops group members in RT to create new saved searches |
[production] |
| 00:11 |
<catrope> |
synchronized php-1.22wmf13/extensions/VisualEditor/modules/ve/ui/tools/buttons/ve.ui.UnderlineButtonTool.js 'touch' |
[production] |
| 00:04 |
<catrope> |
Finished syncing Wikimedia installation... : Updating VisualEditor to master |
[production] |
|
2013-08-20
§
|
| 23:56 |
<catrope> |
Started syncing Wikimedia installation... : Updating VisualEditor to master |
[production] |
| 23:47 |
<mutante> |
powercycling frozen mw1173 |
[production] |
| 23:45 |
<mutante> |
powercycling frozen mw1089 |
[production] |
| 23:42 |
<mutante> |
powercycling srv281 |
[production] |
| 23:38 |
<catrope> |
synchronized php-1.22wmf13/extensions/VisualEditor 'Deploy new VE code to wmf13 first for testing' |
[production] |
| 23:38 |
<RoanKattouw> |
Repeating previously aborted sync-dir of extensions/VisualEditor now that the dsh node list is fixed |
[production] |
| 23:37 |
<mutante> |
manually removed mw1046 from dsh groups mediawiki-installation,apaches,apaches-eqiad on tin |
[production] |
| 23:32 |
<RoanKattouw> |
Forcing puppet on tin to update dsh lists for mw1046 removal |
[production] |
| 23:00 |
<ori-l> |
During sync-dir, SSH timeouts from srv281, mw1089, mw1173; rsync errors on mw1046 (RO fs; previously reported) |
[production] |