|
2018-02-08
ยง
|
| 20:26 |
<ppchelko@tin> |
Started deploy [cpjobqueue/deploy@9adaa92]: Update node-rdkafka to 2.0+. Canary |
[production] |
| 20:24 |
<otto@tin> |
Finished deploy [eventstreams/deploy@7629e16]: (no justification provided) (duration: 00m 22s) |
[production] |
| 20:24 |
<otto@tin> |
Started deploy [eventstreams/deploy@7629e16]: (no justification provided) |
[production] |
| 20:20 |
<ottomata> |
starting deploy process to update scb cluster to librdkafka 0.11 and node-rdkafka 2. we will depool, stop puppet, deploy, test, start puppet on each node |
[production] |
| 20:03 |
<no_justification> |
gerrit: killed about 12 parallel clones of mediawiki/extensions/Math that had been running between 2-3 days (wtf?) |
[production] |
| 19:24 |
<catrope@tin> |
Synchronized php-1.31.0-wmf.20/extensions/Flow/includes/Model/AbstractRevision.php: T186077 (duration: 01m 11s) |
[production] |
| 19:19 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable TemplateStyles on svwiki (T176082) (duration: 01m 11s) |
[production] |
| 19:17 |
<catrope@tin> |
Synchronized php-1.31.0-wmf.20/extensions/Campaigns/CampaignsSecondaryAuthenticationProvider.php: T185870 (duration: 01m 13s) |
[production] |
| 19:02 |
<bsitzmann@tin> |
Finished deploy [mobileapps/deploy@541a7f7]: Update mobileapps to e6fbc94 (duration: 08m 21s) |
[production] |
| 19:00 |
<bblack> |
lvs@ulsfo - all back to normal |
[production] |
| 18:55 |
<bblack> |
lvs@ulsfo - puppet disabled, trying tagged vlan deploy |
[production] |
| 18:54 |
<bsitzmann@tin> |
Started deploy [mobileapps/deploy@541a7f7]: Update mobileapps to e6fbc94 |
[production] |
| 18:38 |
<arlolra> |
Updated Parsoid to 961a5cf (T186630) |
[production] |
| 18:27 |
<arlolra@tin> |
Finished deploy [parsoid/deploy@1367057]: Updating Parsoid to 961a5cf (duration: 08m 11s) |
[production] |
| 18:26 |
<andrew@tin> |
Finished deploy [horizon/deploy@9e9d458]: updating with designate dashboard -- take three (duration: 01m 16s) |
[production] |
| 18:25 |
<andrew@tin> |
Started deploy [horizon/deploy@9e9d458]: updating with designate dashboard -- take three |
[production] |
| 18:19 |
<arlolra@tin> |
Started deploy [parsoid/deploy@1367057]: Updating Parsoid to 961a5cf |
[production] |
| 18:10 |
<ema> |
upgrade cp2026 to varnish 5 |
[production] |
| 17:55 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=dns400[12].wikimedia.org |
[production] |
| 17:21 |
<akosiaris> |
repool sca1004 (zotero) for T181121 |
[production] |
| 17:21 |
<akosiaris@puppetmaster1001> |
conftool action : set/pooled=yes; selector: sca1004.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=sca', 'service=zotero']) |
[production] |
| 17:16 |
<ema> |
upgrade cp2024 to varnish 5 |
[production] |
| 16:58 |
<ema> |
upgrade cp2022 to varnish 5 |
[production] |
| 16:39 |
<moritzm> |
installing PHP7 security updates |
[production] |
| 16:32 |
<moritzm> |
installing mysql security updates on auth* |
[production] |
| 16:31 |
<ema> |
upgrade cp2020 to varnish 5 |
[production] |
| 16:30 |
<bblack> |
puppet disabled on all ntp servers for initial ulsfo recdns/ntp config process |
[production] |
| 16:25 |
<bblack> |
puppet disabled on lvs400[67] for initial ulsfo recdns config process |
[production] |
| 16:23 |
<elukey> |
stop archiva on meitnerium to swap /var/lib/archiva from the root partition to a new separate one - T186020 |
[production] |
| 16:20 |
<akosiaris> |
reboot ganeti1005 T181121 |
[production] |
| 16:18 |
<akosiaris> |
depool sca1004 (zotero) for T181121 |
[production] |
| 16:17 |
<akosiaris@puppetmaster1001> |
conftool action : set/pooled=no; selector: sca1004.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=sca', 'service=zotero']) |
[production] |
| 16:13 |
<bblack> |
rebooting dns400[12] (downtimed, currently spare::system) |
[production] |
| 16:13 |
<ema> |
upgrade cp2017 to varnish 5 |
[production] |
| 16:11 |
<andrew@tin> |
Finished deploy [horizon/deploy@9af532a]: updating with designate dashboard -- take two (duration: 01m 24s) |
[production] |
| 16:10 |
<andrew@tin> |
Started deploy [horizon/deploy@9af532a]: updating with designate dashboard -- take two |
[production] |
| 16:05 |
<bblack> |
ntp servers back to normal |
[production] |
| 16:04 |
<andrew@tin> |
Finished deploy [horizon/deploy@2f176e2]: updating with designate dashboard (duration: 01m 11s) |
[production] |
| 16:03 |
<andrew@tin> |
Started deploy [horizon/deploy@2f176e2]: updating with designate dashboard |
[production] |
| 15:57 |
<ema> |
upgrade cp2014 to varnish 5 |
[production] |
| 15:48 |
<moritzm> |
installing libio-socket-ssl-perl update from jessie point release |
[production] |
| 15:47 |
<bblack> |
disabling puppet on all global dns recursors for controlled config deploy |
[production] |
| 15:35 |
<ema> |
upgrade cp2011 to varnish 5 |
[production] |
| 15:18 |
<ema> |
upgrade cp2008 to varnish 5 |
[production] |
| 15:14 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1073 - T162807 (duration: 01m 12s) |
[production] |
| 14:59 |
<moritzm> |
installing icu security updates from jessie/stretch point releases |
[production] |
| 14:56 |
<ema> |
upgrade cp2005 to varnish 5 |
[production] |
| 14:49 |
<akosiaris> |
migrate all running VMs off ganeti1005 T181121 |
[production] |
| 14:47 |
<anomie@tin> |
Synchronized wmf-config/InitialiseSettings.php: Setting wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on meta and mediawiki.org (duration: 01m 12s) |
[production] |
| 14:43 |
<zeljkof> |
EU SWAT finished |
[production] |