1-50 of 10000 results (21ms)
2020-10-05 ยง
23:11 <ejegg> updated payments staging from 52704ffe24 to db03677b2d [production]
22:27 <mutante> removing shinken puppet module and role [production]
22:01 <ebernhardson> restore wikidatawiki_content enwiki_content enwiki_general and commonswiki_file to default index.merge.policy.deletes_pct_allowed on eqiad cirrus cluster T264053 [production]
21:01 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:59 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:30 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:28 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:26 <ebernhardson> restart elasticsearch_6@production-search-codfw on elastic2051 to take reduced (32 sector, 16kB) readahead settings T264053 [production]
20:13 <ebernhardson> restart elasticsearch_6@production-search-codfw on elastic2051 to take reduced (64 sector, 32kB) readahead settings T264053 [production]
19:56 <ebernhardson> restart elasticsearch_6@production-search-codfw on elastic2050 to take reduced (128kB) readahead settings T264053 [production]
19:31 <mutante> ran sre.dns.netbox to push addition of an-worker1113 which was commited in prod repo but not in netbox data [production]
19:30 <dzahn@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:27 <dzahn@cumin1001> START - Cookbook sre.dns.netbox [production]
18:59 <mforns@deploy1001> Finished deploy [analytics/refinery@2c6c335] (thin): [THIN] Special deployment to unblock deletion jobs [analytics/refinery@2c6c335e61cecd0321ec6f066a153feaf2dbbc27] (duration: 00m 08s) [production]
18:59 <mforns@deploy1001> Started deploy [analytics/refinery@2c6c335] (thin): [THIN] Special deployment to unblock deletion jobs [analytics/refinery@2c6c335e61cecd0321ec6f066a153feaf2dbbc27] [production]
18:58 <mforns@deploy1001> Finished deploy [analytics/refinery@2c6c335]: Special deployment to unblock deletion jobs [analytics/refinery@2c6c335e61cecd0321ec6f066a153feaf2dbbc27] (duration: 12m 08s) [production]
18:46 <mforns@deploy1001> Started deploy [analytics/refinery@2c6c335]: Special deployment to unblock deletion jobs [analytics/refinery@2c6c335e61cecd0321ec6f066a153feaf2dbbc27] [production]
18:17 <elukey@cumin1001> END (FAIL) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=99) [production]
18:17 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
18:15 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) [production]
18:13 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
18:11 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) [production]
18:10 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
17:53 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
17:51 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
17:29 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
17:27 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
17:25 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
17:25 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
17:00 <hnowlan@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
17:00 <hnowlan@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
16:59 <hnowlan@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
16:59 <hnowlan@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
16:51 <hnowlan@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
16:51 <hnowlan@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
15:15 <ppchelko@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
14:56 <ppchelko@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
14:55 <ppchelko@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
14:41 <elukey> shutdown stat1005 and stat1008 for ram expansion (1005 again) [production]
14:36 <ppchelko@deploy1001> Finished deploy [restbase/deploy@366a543]: T263133 T264035 (duration: 22m 23s) [production]
14:25 <elukey> shutdown an-master1001 for ram expansion [production]
14:13 <ppchelko@deploy1001> Started deploy [restbase/deploy@366a543]: T263133 T264035 [production]
14:01 <filippo@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'citoid' for release 'production' . [production]
13:58 <filippo@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'citoid' for release 'production' . [production]
13:55 <filippo@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'citoid' for release 'staging' . [production]
13:54 <elukey> shutdown stat1005 for ram upgrade [production]
13:31 <elukey> shutdown an-master1002 for ram expansion (64 -> 128G) [production]
12:39 <moritzm> installing curl security updates on remaining hosts [production]
11:34 <hoo@deploy1001> Synchronized wmf-config/: Revert "Remove $wgExtraLanguageNames from Wikidata and Commons" (T264295) (duration: 00m 59s) [production]
11:28 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: be73f155001e9095697c3c21a208c63e7bf5d2d1: Move changetags right from users to sysop [trwiki] (T264508) (duration: 00m 59s) [production]