|
2018-10-02
ยง
|
| 21:12 |
<banyek> |
re-enabling and starting backups on host es2001 (TT205257) |
[production] |
| 21:01 |
<gtirloni> |
labstore2003 stopped service block_sync |
[production] |
| 20:15 |
<dduvall@deploy1001> |
Finished scap: group0 to php-1.32.0-wmf.24 (duration: 33m 00s) |
[production] |
| 20:04 |
<Jeff_Green> |
authdns-update to deploy new IP for frbast2001.frack.eqiad.wmnet |
[production] |
| 19:50 |
<XioNoX> |
update prefix-list fundraising-codfw-internal4 to /24 on pfw3-codfw - T204271 |
[production] |
| 19:42 |
<dduvall@deploy1001> |
Started scap: group0 to php-1.32.0-wmf.24 |
[production] |
| 19:36 |
<dduvall@deploy1001> |
Pruned MediaWiki: 1.32.0-wmf.19 (duration: 07m 25s) |
[production] |
| 19:21 |
<XioNoX> |
update fw policies on pfw3-eqiad - T204271 |
[production] |
| 19:19 |
<XioNoX> |
update fw policies on pfw3-codfw - T204271 |
[production] |
| 18:39 |
<XioNoX> |
replace 10.195.0.73/29 with 10.195.0.65/28 on pfw3-codfw - T204271 |
[production] |
| 18:26 |
<XioNoX> |
remove old 10.195.0.65/29 from pfw3-codfw - T204271 |
[production] |
| 18:24 |
<jynus> |
restarting ferm on dbstore2002 T205257 |
[production] |
| 18:07 |
<arlolra> |
Updated Parsoid to 65d6f82 (T163438, T205674, T205673) |
[production] |
| 18:07 |
<ariel@deploy1001> |
Finished deploy [dumps/dumps@a9570fb]: fix incr dumps multiversion conf setting (duration: 00m 06s) |
[production] |
| 18:07 |
<ariel@deploy1001> |
Started deploy [dumps/dumps@a9570fb]: fix incr dumps multiversion conf setting |
[production] |
| 18:01 |
<arlolra@deploy1001> |
Finished deploy [parsoid/deploy@19053a3]: Updating Parsoid to 65d6f82 (duration: 10m 44s) |
[production] |
| 17:51 |
<arlolra@deploy1001> |
Started deploy [parsoid/deploy@19053a3]: Updating Parsoid to 65d6f82 |
[production] |
| 17:37 |
<XioNoX> |
update NAT for frbast2001 on pfw3-codfw - T204271 |
[production] |
| 17:25 |
<XioNoX> |
update fw policies on pfw3-eqiad - T204271 |
[production] |
| 17:22 |
<XioNoX> |
update fw policies on pfw3-codfw - T204271 |
[production] |
| 17:22 |
<andrewbogott> |
upgraded wikitech-static to remotes/origin/REL1_31 |
[production] |
| 17:18 |
<andrewbogott> |
upgrading debian packages and MediaWiki version on wikitech-static |
[production] |
| 16:53 |
<jynus> |
setup test s3 replication channel on db1110 (filtered) |
[production] |
| 16:49 |
<XioNoX> |
assign 10.195.0.129/29 to pfw3-codfw:reth0.2133 - T204271 |
[production] |
| 16:38 |
<cmjohnson1> |
swapping failed disk db1067 T205780 |
[production] |
| 16:04 |
<ppchelko@deploy1001> |
Finished deploy [cpjobqueue/deploy@093551f]: Increase cirrusSearchLinksUpdate concurrency (duration: 01m 06s) |
[production] |
| 16:03 |
<ppchelko@deploy1001> |
Started deploy [cpjobqueue/deploy@093551f]: Increase cirrusSearchLinksUpdate concurrency |
[production] |
| 15:50 |
<marxarelli> |
cutting 1.32.0-wmf.24 branch |
[production] |
| 15:33 |
<gehel> |
cleanup old cronjob (cleanup GC logs) on all elasticsearch servers |
[production] |
| 15:24 |
<akosiaris> |
upgrade mathoid chart version to 0.0.11 |
[production] |
| 15:24 |
<akosiaris@deploy1001> |
scap-helm mathoid finished |
[production] |
| 15:23 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster codfw completed |
[production] |
| 15:23 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster eqiad completed |
[production] |
| 15:23 |
<akosiaris@deploy1001> |
scap-helm mathoid upgrade production stable/mathoid [namespace: mathoid, clusters: eqiad,codfw] |
[production] |
| 15:21 |
<akosiaris@deploy1001> |
scap-helm mathoid finished |
[production] |
| 15:21 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster codfw completed |
[production] |
| 15:21 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster eqiad completed |
[production] |
| 15:21 |
<akosiaris@deploy1001> |
scap-helm mathoid upgrade -h [namespace: mathoid, clusters: eqiad,codfw] |
[production] |
| 14:11 |
<banyek> |
powering off dbstore2002.codfw.wmnet for BBU change (T205257) |
[production] |
| 13:46 |
<marostegui> |
Deploy schema change on s4 eqiad, this will generate lag on eqiad - T205913 |
[production] |
| 13:06 |
<marostegui> |
Deploy schema change on s7 eqiad, this will generate lag on eqiad - T205913 |
[production] |
| 12:47 |
<banyek> |
converting enwiki.content to TokuDB on host dbstrore1002 (T205544) |
[production] |
| 12:47 |
<banyek> |
converting enwiki.contents to TokuDB on host dbstrore1002 (T205544) |
[production] |
| 11:58 |
<banyek> |
converting wikidatawiki.slots to TokuDB on host dbstrore1002 (T205544) |
[production] |
| 11:41 |
<arturo> |
downtime labstore1007 load check in icinga for 1d |
[production] |
| 11:21 |
<zeljkof> |
EU SWAT finished |
[production] |
| 11:19 |
<ladsgroup@deploy1001> |
Synchronized php-1.32.0-wmf.23/extensions/FlaggedRevs/frontend/specialpages/reports/ProblemChanges_body.php: SWAT: [[gerrit:463917|Use proper index on change_tag table (T205904)]] (duration: 00m 57s) |
[production] |
| 10:58 |
<mobrovac@deploy1001> |
Synchronized rpc/RunSingleJob.php: RunSingleJob: Delay job execution while in read-only mode - T204154 (duration: 00m 57s) |
[production] |
| 10:34 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2092 (duration: 00m 56s) |
[production] |
| 10:24 |
<marostegui> |
Deploy schema change on db2092 - T203709 |
[production] |