251-300 of 10000 results (31ms)
2020-01-14 §
06:20 <marostegui> Deploy schema change on s3 master for officewiki and techconductwiki T242688 [production]
06:01 <marostegui> Remove partitions from revision table on db1103:3312 [production]
06:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1103:3312 - T239453', diff saved to https://phabricator.wikimedia.org/P10141 and previous config saved to /var/cache/conftool/dbconfig/20200114-060116-marostegui.json [production]
06:00 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1105:3312 after removing partitions from revision table', diff saved to https://phabricator.wikimedia.org/P10140 and previous config saved to /var/cache/conftool/dbconfig/20200114-060003-marostegui.json [production]
05:29 <andrewbogott> rebooting cloudservices1004 to make sure all my upgrades are sustainable [production]
01:03 <catrope@deploy1001> Synchronized php-1.35.0-wmf.14/extensions/GrowthExperiments/: Various topic search-related cherry-picks (duration: 00m 57s) [production]
2020-01-13 §
21:35 <milimetric@deploy1001> Finished deploy [analytics/refinery@690517c]: Referer Classify change (duration: 09m 08s) [production]
21:32 <arlolra@deploy1001> Finished deploy [parsoid/deploy@dd92eeb]: Updating Parsoid to 5d37da1 (duration: 08m 21s) [production]
21:26 <milimetric@deploy1001> Started deploy [analytics/refinery@690517c]: Referer Classify change [production]
21:24 <arlolra@deploy1001> Started deploy [parsoid/deploy@dd92eeb]: Updating Parsoid to 5d37da1 [production]
20:37 <clarakosi@deploy1001> Finished deploy [restbase/deploy@bfdd342]: Use parsoid_uri, add ngwiki. T241756, T240771 (duration: 15m 41s) [production]
20:21 <clarakosi@deploy1001> Started deploy [restbase/deploy@bfdd342]: Use parsoid_uri, add ngwiki. T241756, T240771 [production]
19:39 <tgr> ran disableOATHAuthForUser.php for T242543 [production]
19:22 <tgr@deploy1001> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:509914|Revert a temporary CommonsMetadata cache validation hook that has been unneeded for a long time]] (duration: 00m 56s) [production]
15:56 <moritzm> installing cyrus-sasl security updates [production]
15:19 <moritzm> remove hassium in Ganeti T224567 [production]
15:19 <akosiaris@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
15:18 <jmm@cumin2001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) [production]
15:18 <akosiaris@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
15:18 <jmm@cumin2001> START - Cookbook sre.hosts.decommission [production]
15:00 <joal@deploy1001> Finished deploy [analytics/hdfs-tools/deploy@a1b4d34]: Deploy hdfs-rsync bug correction (duration: 00m 08s) [production]
15:00 <joal@deploy1001> Started deploy [analytics/hdfs-tools/deploy@a1b4d34]: Deploy hdfs-rsync bug correction [production]
14:58 <jmm@cumin2001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) [production]
14:57 <jmm@cumin2001> START - Cookbook sre.hosts.decommission [production]
14:55 <moritzm> remove hassaleh in Ganeti T224567 [production]
14:24 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:558017| Bumping portals to master (563985)]] (duration: 00m 55s) [production]
14:24 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:558017| Bumping portals to master (563985)]] (duration: 00m 56s) [production]
13:11 <moritzm> upgrade mw canaries to PHP 7.2.26 T241222 [production]
12:08 <Urbanecm> EU SWAT done [production]
12:08 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: c7cf53c: Deploy partial blocks on enwiki (T242569) (duration: 00m 55s) [production]
11:58 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:558017| Bumping portals to master (563985)]] (duration: 00m 55s) [production]
11:57 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:558017| Bumping portals to master (563985)]] (duration: 00m 55s) [production]
11:42 <moritzm> upgrading remaining mwdebug* servers and mw1261 to PHP 7.2.26 T241222 [production]
11:04 <volans@deploy1001> Finished deploy [debmonitor/deploy@265059b]: Release v0.2.3 (duration: 01m 10s) [production]
11:03 <volans@deploy1001> Started deploy [debmonitor/deploy@265059b]: Release v0.2.3 [production]
10:51 <vgutierrez> pooling esams for ncredir - T242321 [production]
09:38 <moritzm> rename Ganeti group in ulsfo from "default" to "row_1" [production]
09:16 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:16 <filippo@cumin1001> START - Cookbook sre.hosts.downtime [production]
07:53 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1112', diff saved to https://phabricator.wikimedia.org/P10134 and previous config saved to /var/cache/conftool/dbconfig/20200113-075334-marostegui.json [production]
07:36 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1112', diff saved to https://phabricator.wikimedia.org/P10133 and previous config saved to /var/cache/conftool/dbconfig/20200113-073656-marostegui.json [production]
07:30 <XioNoX> cr3-knams> clear bfd session fe80::5e5e:ab00:d3d:85c - T240659 [production]
07:26 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1112', diff saved to https://phabricator.wikimedia.org/P10132 and previous config saved to /var/cache/conftool/dbconfig/20200113-072611-marostegui.json [production]
06:45 <marostegui> Upgrade db1112 [production]
06:36 <marostegui> Deploy schema change on db1112 with replication (lag will appear on s3 on labs) - T234052 [production]
06:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1112', diff saved to https://phabricator.wikimedia.org/P10131 and previous config saved to /var/cache/conftool/dbconfig/20200113-063513-marostegui.json [production]
06:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1081 for compression T232446', diff saved to https://phabricator.wikimedia.org/P10130 and previous config saved to /var/cache/conftool/dbconfig/20200113-062007-marostegui.json [production]
06:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1084', diff saved to https://phabricator.wikimedia.org/P10129 and previous config saved to /var/cache/conftool/dbconfig/20200113-061835-marostegui.json [production]
06:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1084 after compression', diff saved to https://phabricator.wikimedia.org/P10128 and previous config saved to /var/cache/conftool/dbconfig/20200113-061434-marostegui.json [production]
06:11 <marostegui> Deploy schema change on s1 master (db1083) - T234052 [production]