|
2018-08-21
§
|
| 16:32 |
<XioNoX> |
enabling ae1 on asw-a-eqiad then ae1 on cr1-eqiad - T202075 |
[production] |
| 16:27 |
<godog> |
upgrade hp raid firmware on ms-be2020 - T141756 |
[production] |
| 16:21 |
<XioNoX> |
disable ae1 on asw-a-eqiad - T202075 |
[production] |
| 16:18 |
<XioNoX> |
disable ae1 on cr1-eqiad - T202075 |
[production] |
| 16:17 |
<papaul> |
replacing PEM2 on cr2-codfw |
[production] |
| 15:38 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1096 (duration: 00m 50s) |
[production] |
| 15:24 |
<ejegg> |
restarted CiviCRM deduplication jobs |
[production] |
| 15:10 |
<moritzm> |
installing apache updates from stretch 9.5 point release |
[production] |
| 14:58 |
<moritzm> |
rolling restart of proton* to pick up nodejs security update |
[production] |
| 13:44 |
<jynus> |
stop db1096 (both s5 an s6) for upgrade |
[production] |
| 13:36 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1096 (duration: 00m 49s) |
[production] |
| 13:27 |
<jynus> |
stop db2090 for upgrade |
[production] |
| 12:38 |
<ema> |
power-cycle ms-be2020: multiple alerts, no ssh access, I/O errors in console |
[production] |
| 12:34 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1100 (duration: 00m 50s) |
[production] |
| 12:31 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool es1015 (duration: 00m 50s) |
[production] |
| 12:18 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1100 (duration: 00m 49s) |
[production] |
| 12:14 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1097:3315 (duration: 00m 49s) |
[production] |
| 12:13 |
<moritzm> |
rolling restart of services in scb/eqiad to pick up the nodejs security update |
[production] |
| 12:05 |
<jynus> |
stop es1015 for upgrade |
[production] |
| 11:59 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool es1015 (duration: 00m 50s) |
[production] |
| 11:37 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1097:3315 (duration: 00m 50s) |
[production] |
| 11:16 |
<zeljkof> |
EU SWAT finished |
[production] |
| 11:16 |
<zfilipin@deploy1001> |
Synchronized wmf-config/Wikibase-production.php: SWAT: [[gerrit:449017|Update several Wikidata-related configs]] (duration: 00m 50s) |
[production] |
| 11:05 |
<zfilipin@deploy1001> |
Synchronized wmf-config/throttle.php: SWAT: [[gerrit:454186|Add IP ranges for ptWiki editathon (T202354)]] (duration: 00m 52s) |
[production] |
| 09:58 |
<moritzm> |
installing libxcursor security updates |
[production] |
| 09:46 |
<moritzm> |
installing libxml2 security updates on trusty |
[production] |
| 09:27 |
<moritzm> |
installing mutt security updates |
[production] |
| 09:16 |
<moritzm> |
installing jetty9 security updates |
[production] |
| 08:37 |
<marostegui> |
Rename blob_tracking and blob_orphans on db1089 - T59186 |
[production] |
| 07:55 |
<marostegui> |
Slowly reload haproxies on dbproxy10XX - T201021 |
[production] |
| 06:27 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1096:3315 (duration: 00m 50s) |
[production] |
| 05:12 |
<marostegui> |
Stop puppet on dbproxy1006 to do some haproxy logging tests - https://phabricator.wikimedia.org/T201021 |
[production] |
| 05:09 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1096:3315 (duration: 00m 51s) |
[production] |
| 04:54 |
<marostegui> |
Set max_connections back from 800 to 500 on db1073 - T188589 |
[production] |
| 02:47 |
<krinkle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: rm unused Id5f5295d (duration: 00m 50s) |
[production] |
| 02:43 |
<krinkle@deploy1001> |
Synchronized wmf-config/CommonSettings.php: rm unused I741b16452 (duration: 00m 56s) |
[production] |
| 02:39 |
<l10nupdate@deploy1001> |
ResourceLoader cache refresh completed at Tue Aug 21 02:39:12 UTC 2018 (duration 10m 18s) |
[production] |
| 02:28 |
<l10nupdate@deploy1001> |
scap sync-l10n completed (1.32.0-wmf.16) (duration: 08m 32s) |
[production] |
| 01:35 |
<aaron@deploy1001> |
Synchronized wmf-config/mc.php: Enable broadcasted mcrouter operations for all wikis (duration: 00m 51s) |
[production] |
| 00:07 |
<ppchelko@deploy1001> |
Finished deploy [cpjobqueue/deploy@9fbf9fa]: Don't clean up the queue size counters (duration: 00m 46s) |
[production] |
| 00:07 |
<ppchelko@deploy1001> |
Started deploy [cpjobqueue/deploy@9fbf9fa]: Don't clean up the queue size counters |
[production] |
|
2018-08-20
§
|
| 23:09 |
<twentyafterfour> |
Finished evening SWAT |
[production] |
| 23:08 |
<twentyafterfour@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/454083/ Phab Task: T202065 (duration: 00m 51s) |
[production] |
| 23:06 |
<twentyafterfour> |
https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/454083/ deployed to mwdebug1001 |
[production] |
| 22:53 |
<ppchelko@deploy1001> |
Finished deploy [cpjobqueue/deploy@c947567]: Use timing for queue sizes as a workaround to grafana not plotting gauges (duration: 00m 45s) |
[production] |
| 22:52 |
<ppchelko@deploy1001> |
Started deploy [cpjobqueue/deploy@c947567]: Use timing for queue sizes as a workaround to grafana not plotting gauges |
[production] |
| 22:29 |
<eileen> |
civicrm revision changed from 3f6dd6c9a6 to 8910c34502, config revision is d787269882 |
[production] |
| 22:27 |
<ppchelko@deploy1001> |
Finished deploy [cpjobqueue/deploy@541932a]: Make the queue sized report stats once a second T202107 (duration: 00m 46s) |
[production] |
| 22:26 |
<ppchelko@deploy1001> |
Started deploy [cpjobqueue/deploy@541932a]: Make the queue sized report stats once a second T202107 |
[production] |
| 22:13 |
<mutante> |
ganeti1001 - creating 3 new VMs for analytics-tools |
[production] |