|
2013-10-21
§
|
| 23:17 |
<mwalker> |
CentralNotice deploy had an issue where we are now pushing ALL traffic to the mobile site for CN -- ganglia reports a spike and I'm now reverting the change |
[production] |
| 23:09 |
<^demon|lunch> |
elastic: index created for bnwiki, running force indexing in 4 processes on terbium |
[production] |
| 23:07 |
<demon> |
synchronized wmf-config/InitialiseSettings.php 'bnwiki gets cirrus as secondary' |
[production] |
| 23:05 |
<mwalker> |
synchronized php-1.22wmf22/extensions/CentralNotice/ 'Updating CentralNotice to master' |
[production] |
| 23:05 |
<mwalker> |
synchronized php-1.22wmf21/extensions/CentralNotice/ 'Updating CentralNotice to master' |
[production] |
| 22:49 |
<paravoid> |
powercycling ms-be1004, locked up(?) |
[production] |
| 22:13 |
<maxsem> |
synchronized php-1.22wmf22/extensions/MobileFrontend/ 'https://gerrit.wikimedia.org/r/91065' |
[production] |
| 22:04 |
<AaronSchulz> |
Running syncFilebackend on all wikis (should reduce sync errors; a few a popping up in the logs) |
[production] |
| 21:34 |
<aaron> |
synchronized wmf-config/filebackend.php 'Added eqiad swift to multiwrite backends' |
[production] |
| 21:06 |
<cmjohnson1> |
morebots are you working? |
[production] |
| 21:04 |
<cmjohnson1> |
dns update |
[production] |
| 20:05 |
<cmjohnson1> |
dns update |
[production] |
| 19:38 |
<cmjohnson1> |
dns update |
[production] |
| 18:40 |
<reedy> |
synchronized wmf-config/interwiki.cdb 'Updating interwiki cache' |
[production] |
| 18:34 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.22wmf22 |
[production] |
| 18:29 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 17:44 |
<cmjohnson1> |
reclaiming cp1021-36, cp1041-42 per RT5981 |
[production] |
| 17:24 |
<Reedy> |
The @Wikimedia Operations team is seeking proposals for a new datacenter in the midwestern/western US: https://blog.wikimedia.org/2013/10/21/rfp-new-datacenter-continental-us/ |
[production] |
| 16:30 |
<ori-l> |
ran sync-apache for I1113b9594 & Ie73ce6213; verified on mw60; gracefuling. |
[production] |
| 16:26 |
<demon> |
synchronized wmf-config/CirrusSearch-production.php 'LVS for cirrus on all wikis' |
[production] |
| 16:11 |
<demon> |
synchronized wmf-config/CirrusSearch-production.php 'LVS for cirrus on test2wiki' |
[production] |
| 16:07 |
<RobH> |
updated akismet blog plugin |
[production] |
| 16:03 |
<RobH> |
tweaked permissions of the blog's w3 caching plugin to actually make it work again |
[production] |
| 11:47 |
<mark> |
Shutdown csw2-esams:xe-2/1/1 (1 DF leg) |
[production] |
| 11:31 |
<hashar> |
Gerrit replication have been broken since Oct 19th roughly 20:50 UTC. {{bug|55948}} |
[production] |
| 10:43 |
<hashar> |
Gerrit replication is broken, side effects: git.wm.o shows outdated trees and Jenkins might be missing some commits |
[production] |
| 09:16 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 09:16 |
<reedy> |
synchronized docroot/bits/apple-touch/ |
[production] |
| 09:03 |
<hashar> |
restarting Zuul |
[production] |
| 09:02 |
<hashar> |
Jenkins restarted / upgraded |
[production] |
| 08:59 |
<hashar> |
rerestarting Jenkins. |
[production] |
| 08:51 |
<akosiaris> |
forced verified +2 on gerrit 90874 since jenkins is being upgraded |
[production] |
| 08:47 |
<hashar> |
jenkins: upgrading plugins |
[production] |
| 08:43 |
<hashar> |
stopping Zuul again. Need to upgrade Jenkins plugins |
[production] |
| 08:42 |
<hashar> |
restarted Zuul |
[production] |
| 08:00 |
<hashar> |
stopping Zuul / Jenkins |
[production] |
| 07:59 |
<hashar> |
upgrading Jenkins for scheduled maintenance |
[production] |
| 06:15 |
<apergos> |
moved older swift replication logs from copper:/root to iron:/root/swift-repl/ (now gzipped), copper was full |
[production] |
| 03:21 |
<Tim> |
on mw1109: stopped apache to test cgconfig |
[production] |
| 02:39 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Mon Oct 21 02:39:00 UTC 2013 |
[production] |
| 02:19 |
<LocalisationUpdate> |
completed (1.22wmf22) at Mon Oct 21 02:19:25 UTC 2013 |
[production] |
| 02:10 |
<LocalisationUpdate> |
completed (1.22wmf21) at Mon Oct 21 02:10:47 UTC 2013 |
[production] |
| 00:38 |
<^d> |
bringing gerrit down to troubleshoot replication |
[production] |