|
2017-11-16
ยง
|
| 19:05 |
<demon@tin> |
Unlocked for deployment [ALL REPOSITORIES]: Dealing with outage, no deploys for now (duration: 01m 22s) |
[production] |
| 19:03 |
<demon@tin> |
Locking from deployment [ALL REPOSITORIES]: Dealing with outage, no deploys for now (planned duration: 60m 00s) |
[production] |
| 18:48 |
<marostegui> |
dewikipedia and wikidata currently back to writable |
[production] |
| 18:44 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Pool only 'good' servers (duration: 00m 48s) |
[production] |
| 18:41 |
<mutante> |
dewikipedia and wikidata currently back to readonly-mode while wikidata is being worked on |
[production] |
| 18:41 |
<marostegui> |
Set s5 master read_only |
[production] |
| 18:25 |
<akosiaris> |
set mw2251 to inactive. T180724 |
[production] |
| 18:24 |
<akosiaris@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: name=mw2251.codfw.wmnet |
[production] |
| 17:57 |
<gehel> |
silencing wdqs |
[production] |
| 17:55 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Failover db1063 to db1070 (duration: 00m 46s) |
[production] |
| 17:44 |
<XioNoX> |
merging netbox CR |
[production] |
| 17:41 |
<bblack> |
disable port ge-5/0/39 on asw-c-eqiad (db1063) |
[production] |
| 17:40 |
<urandom> |
Decommissioning Cassandra, restbase1014-b.eqiad.wmnet (T179422) |
[production] |
| 17:36 |
<jynus> |
stopping slave on db1070 |
[production] |
| 17:28 |
<urandom> |
Converting 'enwiki parsoid' to size-tiered compaction (T179422) |
[production] |
| 17:19 |
<demon@tin> |
Finished scap: consistency (duration: 25m 12s) |
[production] |
| 17:18 |
<bblack> |
Temporary GeoDNS routing changes (eqsin traffic simulation using ulsfo) - https://gerrit.wikimedia.org/r/#/c/391357/ - expecting ~24h, West Asia latencies will probably increase, spike in cache misses, etc... |
[production] |
| 17:18 |
<urandom> |
Converting 'wikipedia parsoid' to size-tiered compaction (T179422) |
[production] |
| 17:16 |
<godog> |
reimage restbase2006 - T179422 |
[production] |
| 17:15 |
<urandom> |
Converting 'commons mobile' to size-tiered compaction (T179422) |
[production] |
| 17:05 |
<urandom> |
Converting 'others mobile' to size-tiered compaction (T179422) |
[production] |
| 16:54 |
<demon@tin> |
Started scap: consistency |
[production] |
| 16:54 |
<godog> |
reimage restbase2006 - T179422 |
[production] |
| 16:48 |
<jynus> |
stop and restart db1071 for upgrade and reconfiguration |
[production] |
| 16:48 |
<herron> |
beginning gradual cutover of codfw mw systems to puppet 4 master puppetmaster2001 |
[production] |
| 16:48 |
<godog> |
upgrade hpsa firmware to 6.06 on restbase2006 - T141756 |
[production] |
| 16:43 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Restore db1034 and db1079 original weight (duration: 00m 48s) |
[production] |
| 16:29 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: mariadb: Depool db1071, pool db1104 as api (duration: 00m 49s) |
[production] |
| 16:21 |
<moritzm> |
restarting apache on labs puppet masters to pick up openssl updates |
[production] |
| 16:06 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1034 weight (duration: 00m 48s) |
[production] |
| 15:47 |
<moritzm> |
rebooting ores1* for update to 4.9.51 |
[production] |
| 15:41 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1034 weight (duration: 00m 49s) |
[production] |
| 15:33 |
<moritzm> |
rebooting seaborgium (slapd) for update to 4.9.51 |
[production] |
| 15:22 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2038 - T178359 (duration: 00m 49s) |
[production] |
| 15:16 |
<milimetric@tin> |
Finished deploy [analytics/refinery@4ef15d3]: Mainly deploying the interlanguage navigation dataset (duration: 14m 29s) |
[production] |
| 15:13 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1034 weight (duration: 00m 48s) |
[production] |
| 15:10 |
<godog> |
upgrade prometheus-redis-exporter to 0.13-1 - T148637 |
[production] |
| 15:06 |
<moritzm> |
installing postgres security updates on labsdb1006/1007 |
[production] |
| 15:02 |
<milimetric@tin> |
Started deploy [analytics/refinery@4ef15d3]: Mainly deploying the interlanguage navigation dataset |
[production] |
| 15:01 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1051 - T177208 (duration: 00m 48s) |
[production] |
| 15:00 |
<herron> |
beginning cut over of codfw canary appservers to puppet 4 master puppetmaster2001 |
[production] |
| 14:55 |
<moritzm> |
rebooting serpens (slapd) for update to 4.9.51 |
[production] |
| 14:50 |
<elukey> |
updating puppet compiler's facts (following https://wikitech.wikimedia.org/w/index.php?title=Nova_Resource:Puppet3-diffs#FAQ) |
[production] |
| 14:41 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1034 weight - T178359 (duration: 00m 49s) |
[production] |
| 14:26 |
<dcausse> |
EU swat done |
[production] |
| 14:23 |
<moritzm> |
rebooting kubetcd* to 4.9.51 |
[production] |
| 14:18 |
<dcausse@tin> |
Synchronized wmf-config/CirrusSearch-common.php: T177913: [cirrus] Add overridden iw prefix for svwiki (2/2) (duration: 00m 48s) |
[production] |
| 14:16 |
<dcausse@tin> |
Synchronized wmf-config/InitialiseSettings.php: T177913: [cirrus] Add overridden iw prefix for svwiki (1/2) (duration: 00m 50s) |
[production] |
| 14:02 |
<gehel> |
starting upgrade of elasticsearch eqiad - T178411 |
[production] |
| 13:46 |
<moritzm> |
rebooting bohrium (piwik host) for update to 4.9.51 |
[production] |