4451-4500 of 10000 results (69ms)
2019-11-15 §
04:17 <vgutierrez> Move cp3062 from nginx to ats-tls - T231627 [production]
04:00 <vgutierrez> Move cp3060 from nginx to ats-tls - T231627 [production]
01:35 <tstarling@deploy1001> Synchronized php-1.35.0-wmf.5/includes/Rest/Handler/CompareHandler.php: deploying REST compare section feature because iOS team need it for a beta release due very soon (duration: 00m 53s) [production]
01:33 <tstarling@deploy1001> Synchronized php-1.35.0-wmf.5/includes/Rest/coreRoutes.json: deploying REST compare section feature because iOS team need it for a beta release due very soon (duration: 00m 52s) [production]
01:32 <tstarling@deploy1001> Synchronized php-1.35.0-wmf.5/includes/parser/Parser.php: deploying REST compare section feature because iOS team need it for a beta release due very soon (duration: 00m 54s) [production]
2019-11-14 §
23:03 <mutante> restarting gerrit to ncrease defaultThreadPoolSize to 2 [production]
22:29 <eileen> civicrm revision changed from a3714003ff to ae9b3819cd, config revision is 6adc66a20b [production]
21:32 <ssastry@deploy1001> Finished deploy [parsoid/deploy@150f9af]: Updating Parsoid to 74203415 (duration: 08m 21s) [production]
21:24 <ssastry@deploy1001> Started deploy [parsoid/deploy@150f9af]: Updating Parsoid to 74203415 [production]
21:14 <gehel@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
20:06 <cdanis@cumin2001> dbctl commit (dc=all): 'remove now-defunct wikitech section T233236', diff saved to https://phabricator.wikimedia.org/P9639 and previous config saved to /var/cache/conftool/dbconfig/20191114-200649-cdanis.json [production]
20:04 <gehel> reloading data on wdqs1004 from wdqs1007 to catch up on lag faster - T238229 [production]
19:57 <gehel@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
19:33 <otto@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'logging-external' . [production]
19:31 <otto@deploy1001> helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'logging-external' . [production]
19:20 <otto@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'logging-external' . [production]
18:49 <catrope@deploy1001> Synchronized wmf-config/: Use s10/s11 dblists for wikitechs (for real this time) (T233236) (duration: 00m 52s) [production]
18:37 <catrope@deploy1001> Synchronized dblists/: Use s10/s11 dblists for wikitechs (T233236) (duration: 00m 51s) [production]
18:35 <catrope@deploy1001> Synchronized dblists/: Add s10/s11 dblists for wikitechs (T233236) (duration: 00m 52s) [production]
18:34 <mutante> scandium - restart php7.2-fpm [production]
18:31 <mutante> phabricator (phab1003, prod server) - upgrade PHP version to 7.2.24 (T237239) [production]
18:17 <cdanis@cumin2001> dbctl commit (dc=all): 'alias wikitech section to new s10 section T233236', diff saved to https://phabricator.wikimedia.org/P9638 and previous config saved to /var/cache/conftool/dbconfig/20191114-181732-cdanis.json [production]
17:46 <robh> running dell epsa tool on cp3056 per T236497 [production]
17:35 <akosiaris@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
17:35 <akosiaris@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . [production]
17:35 <akosiaris@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
17:22 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
17:22 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . [production]
17:22 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
17:22 <ejegg> updated payments-wiki from bd907656fb to 30579d34d8 [production]
17:17 <akosiaris@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
17:17 <akosiaris@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . [production]
17:17 <akosiaris@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
16:09 <mutante> phab2001 - upgrading PHP version to 7.2.24 (T237239) [production]
16:06 <mutante> scandium - upgrading PHP version to 7.2.24 (fyi, @subbu T228069) (T237239) [production]
16:04 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.5/extensions/Wikibase: [[gerrit:550828|Put a layer of APC cache on top of reading wb_terms in SqlEntityInfoBuilder]] (T231011 T229407 T236681), Try II (duration: 00m 56s) [production]
14:54 <ema> pool cp3060 with ATS backend T227432 [production]
14:53 <mholloway-shell@deploy1001> Synchronized php-1.35.0-wmf.5/extensions/MachineVision: Fix bug when when looking up entity for an unknown ID (duration: 00m 53s) [production]
14:48 <anomie@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Set MCR migration stage to NEW on group1 for T198312 (duration: 00m 53s) [production]
14:27 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:24 <ema@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:01 <ema> depool cp3060 and reimage as text_ats T227432 [production]
13:37 <ladsgroup@deploy1001> scap failed: average error rate on 7/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) [production]
13:35 <gehel> depool wdqs1004 to allow catching up on lag - T238229 [production]
13:06 <bblack> removing digicert-2019 files from cache nodes - https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/550829/ [production]
12:24 <mobrovac@deploy1001> Finished deploy [restbase/deploy@58cf5ae]: Fix /metrics/mediarequests/top/ indentation (duration: 14m 52s) [production]
12:09 <mobrovac@deploy1001> Started deploy [restbase/deploy@58cf5ae]: Fix /metrics/mediarequests/top/ indentation [production]
11:58 <mobrovac@deploy1001> Finished deploy [restbase/deploy@58cf5ae] (dev-cluster): Fix /metrics/mediarequests/top/ indentation (duration: 02m 50s) [production]
11:55 <mobrovac@deploy1001> Started deploy [restbase/deploy@58cf5ae] (dev-cluster): Fix /metrics/mediarequests/top/ indentation [production]
11:26 <gehel@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]