251-300 of 10000 results (68ms)
2019-07-30 ยง
18:15 <brennen@deploy1001> Finished scap: testwiki to php-1.34.0-wmf.16 and rebuild l10n cache (duration: 18m 23s) [production]
18:14 <XioNoX> bump cr3-ulsfo<->cr2-eqord ospf metric [production]
18:07 <XioNoX> deactivate transit BGP groups on cr3-ulsfo [production]
18:06 <XioNoX> failover VRRP master to cr4-ulsfo [production]
17:56 <brennen@deploy1001> Started scap: testwiki to php-1.34.0-wmf.16 and rebuild l10n cache [production]
17:55 <brennen@deploy1001> Pruned MediaWiki: 1.34.0-wmf.11 (duration: 07m 40s) [production]
17:53 <bsitzmann@deploy1001> Finished deploy [mobileapps/deploy@af8b471]: Update mobileapps to ec865a7 (duration: 05m 45s) [production]
17:47 <bsitzmann@deploy1001> Started deploy [mobileapps/deploy@af8b471]: Update mobileapps to ec865a7 [production]
17:20 <XioNoX> depool ulsfo for routers upgrades - T227886 [production]
17:15 <godog> use wezen.codfw.wmnet instead of syslog.codfw.wmnet for production hosts [production]
17:00 <thcipriani> gerrit restart incoming -- gc time increasing causing timeouts [production]
16:46 <XioNoX> adding port 9105 to term prometheus in filter labs-in4 - T225296 [production]
16:41 <cdanis@deploy1001> Synchronized wmf-config/etcd.php: Icf57a2ab enable dbctl on all mw canaries (duration: 00m 47s) [production]
16:37 <brennen> cutting 1.34-wmf.16 [production]
16:33 <elukey@cumin1001> START - Cookbook sre.kafka.roll-restart-brokers [production]
16:22 <godog> bounce rsyslog on centrallog1001 - T199406 [production]
15:41 <elukey@cumin1001> END (FAIL) - Cookbook sre.kafka.roll-restart-brokers (exit_code=99) [production]
15:28 <legoktm@deploy1001> Finished scap: Rebuild l10n cache for SecureLinkFixer message (duration: 18m 51s) [production]
15:21 <elukey@cumin1001> START - Cookbook sre.kafka.roll-restart-brokers [production]
15:18 <jijiki> Disable puppet on mw1347 and mw2136, depool and pool back - T219150 [production]
15:13 <elukey> remove snakebite from buster-wikimedia (not needed anymore) [production]
15:09 <legoktm@deploy1001> Started scap: Rebuild l10n cache for SecureLinkFixer message [production]
15:06 <legoktm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable SecureLinkFixer everywhere (T200751) (duration: 00m 47s) [production]
14:47 <cdanis@deploy1001> Synchronized wmf-config/etcd.php: I17c55428 dbctl canary on mwdebug*, mw1261, mw1276 (duration: 00m 47s) [production]
14:36 <cdanis@deploy1001> Synchronized wmf-config/CommonSettings.php: Ie98a8d9e dbctl canary on mwdebug1001 (duration: 00m 47s) [production]
14:34 <cdanis@deploy1001> Synchronized wmf-config/etcd.php: Ie98a8d9e dbctl canary on mwdebug1001 (duration: 00m 47s) [production]
14:33 <cdanis@deploy1001> Synchronized docroot/noc/db.php: Ie98a8d9e dbctl canary on mwdebug1001 (duration: 00m 48s) [production]
14:14 <fsero> refreshing calico policy from code in eqiad [production]
14:13 <fsero> refreshing calico policy from code in codfw [production]
13:38 <marostegui> Move db2094:3315 from db2066 to db2128 - T228258 [production]
13:14 <@> helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . [production]
13:13 <@> helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . [production]
12:36 <marostegui@cumin1001> dbctl commit of MediaWiki config (dc=all), diff saved to 'https://phabricator.wikimedia.org/P8824', previous config saved to /var/cache/conftool/dbconfig/20190730-123630-marostegui.json [production]
12:21 <@> helmfile [EQIAD] Ran 'apply' command on namespace 'termbox' for release 'production' . [production]
12:15 <@> helmfile [CODFW] Ran 'apply' command on namespace 'termbox' for release 'production' . [production]
12:13 <@> helmfile [STAGING] Ran 'apply' command on namespace 'termbox' for release 'test' . [production]
12:13 <jbond42> while testing some changes on the puppet master a bad config caused a small blip in catalouge compilation [production]
12:09 <@> helmfile [STAGING] Ran 'apply' command on namespace 'termbox' for release 'staging' . [production]
11:34 <@> helmfile [EQIAD] Ran 'apply' command on namespace 'cxserver' for release 'production' . [production]
11:31 <@> helmfile [CODFW] Ran 'apply' command on namespace 'cxserver' for release 'production' . [production]
11:30 <jijiki> Depool mw1348 and pool back [production]
11:28 <@> helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . [production]
09:49 <elukey> upload python-snakebite to buster-wikimedia (rebuilt for buster from source) [production]
09:31 <akosiaris@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
09:27 <elukey> add thirdparty/cloudera to buster-wikimedia and import packages to it (pull from the jessie component) [production]
08:17 <marostegui> Stop MySQL on db2038 T227565 [production]
08:10 <marostegui> Remove db2038 from tendril and zarcillo T227565 [production]
08:04 <akosiaris> delete orespoolcounter{1,2}00{1,2} T227640 [production]
08:04 <akosiaris> revoke and deactivate orespoolcounter{1,2}00{1,2} T227640 [production]
07:30 <godog> bounce hhvm on mw1221 [production]