|
2013-03-14
§
|
| 17:49 |
<aaron> |
synchronized wmf-config 'Enabled redis job aggregator fully' |
[production] |
| 17:47 |
<cmjohnson1> |
labsdb1001 powering down |
[production] |
| 17:42 |
<aaron> |
synchronized wmf-config 'Set redis job aggregator on for testwiki.' |
[production] |
| 17:27 |
<reedy> |
synchronized php-1.21wmf11/extensions/Math/ |
[production] |
| 17:23 |
<binasher> |
shutting down mysql on db1009, upgrading to mariadb |
[production] |
| 16:59 |
<cmjohnson1> |
powering down labsdb1002 and labsdb1003 |
[production] |
| 16:59 |
<asher> |
synchronized wmf-config/db-eqiad.php 'pulling db1009 for mariadb migration' |
[production] |
| 16:57 |
<cmjohnson1> |
addin mw1041 back to dsh groups |
[production] |
| 16:26 |
<mlitn> |
synchronized wmf-config 'Prepare AFTv5 config for deployment new features' |
[production] |
| 13:15 |
<mark> |
Set OSPF cost to 10 on csw1-esams:ve7 to facilitate ip multipath |
[production] |
| 13:15 |
<mark> |
Fixed OSPF on cr2-knams:xe-0/0/0 - csw1-esams:e8/2 (earlier) |
[production] |
| 12:16 |
<hashar> |
gallium setting up /srv/ as a copy of integration/docroot.git |
[production] |
| 09:03 |
<apergos> |
restarted varnishncsa.vanadium on niobium, it was using 9gb memory or so |
[production] |
| 02:25 |
<LocalisationUpdate> |
completed (1.21wmf11) at Thu Mar 14 02:25:19 UTC 2013 |
[production] |
| 02:20 |
<hashar> |
jenkins: made Zuul to block changes on operations/debs/ircecho whenever pep8/pyflakes fails. |
[production] |
| 01:35 |
<mutante> |
running puppet on celsus to investigate puppet freshness reports from icinga - it started parsoid, but also celsus is a Wikimedia DECOMMISSIONED server (base::decommissioned). |
[production] |
| 01:10 |
<reedy> |
synchronized php-1.21wmf11/cache/l10n/ |
[production] |
| 00:59 |
<reedy> |
synchronized php-1.21wmf11/extensions/Scribunto |
[production] |
| 00:53 |
<reedy> |
synchronized wmf-config/ |
[production] |
|
2013-03-13
§
|
| 23:43 |
<mutante> |
installing package upgrades on zirconium |
[production] |
| 23:41 |
<demon> |
Finished syncing Wikimedia installation... : |
[production] |
| 23:38 |
<Ryan_Lane> |
deploying change 53464 to OpenStackManager on wikitech |
[production] |
| 23:23 |
<RobH> |
no one sync apaches, we wre doing crazy magic on them |
[production] |
| 23:17 |
<demon> |
Started syncing Wikimedia installation... : |
[production] |
| 23:05 |
<py> |
synchronized wmf-config/lucene-production.php 'moving all search traffic back to eqiad. upgrades over.' |
[production] |
| 23:00 |
<py> |
synchronized wmf-config/lucene-production.php 'moving all search traffic back to eqiad. upgrades over.' |
[production] |
| 22:53 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 22:44 |
<demon> |
synchronized php-1.21wmf11/extensions/Scribunto/ 'Updating Scribunto to master' |
[production] |
| 22:30 |
<robh> |
gracefulled all apaches |
[production] |
| 21:46 |
<awight> |
payments cluster updated to 61d3f1f |
[production] |
| 21:43 |
<pp-pdf1> |
upgraded mwlib to 0.15.3 |
[production] |
| 21:38 |
<Tim> |
on professor: restarted collector |
[production] |
| 21:21 |
<RobH> |
repooling all ssl servers (except broken ssl3004) in all sites. new unified certificate is now in use. |
[production] |
| 21:15 |
<RobH> |
depooling ssl servers 3 and 4 in each dc site |
[production] |
| 21:14 |
<RobH> |
repooling the first two ssl servers per site |
[production] |
| 21:08 |
<py> |
synchronized wmf-config/lucene-production.php 'temp moving all earch traffic to pmtpa for upgrades in eqiad' |
[production] |
| 21:05 |
<RobH> |
setting first 2 ssl servers per dc site to false in pybal |
[production] |
| 20:21 |
<Ryan_Lane> |
repooling ssl1001 |
[production] |
| 20:20 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Re-enable collection' |
[production] |
| 20:16 |
<Ryan_Lane> |
depooling ssl1001 |
[production] |
| 20:08 |
<LeslieCarr> |
cp1022 unresponsive, power cycling |
[production] |
| 19:44 |
<LeslieCarr> |
switched preferred path between esams and eqiad |
[production] |
| 19:41 |
<LeslieCarr> |
deploying a possibly mobile front end traffic effecting ipv6 apache request firewall filter |
[production] |
| 19:26 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Disable Collection extension' |
[production] |
| 19:08 |
<LeslieCarr> |
site was popedotted ! |
[production] |
| 18:47 |
<py> |
synchronized wmf-config/lucene-production.php 'moving all search traffic back to eqaid' |
[production] |
| 18:31 |
<py> |
synchronized wmf-config/lucene-production.php 'temp moving all earch traffic to pmtpa for upgrades in eqiad' |
[production] |
| 18:15 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Enable scribunto EVERYWHERE' |
[production] |
| 18:09 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Everything else to 1.21wmf11 |
[production] |
| 17:23 |
<pp-pdf1> |
downgrade to 0.14.3 |
[production] |