3651-3700 of 10000 results (84ms)
2019-11-12 ยง
19:41 <anomie@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Set MCR migration stage to NEW on group0 for T198312 (duration: 00m 52s) [production]
19:18 <arlolra> Updated Parsoid to 6a0a708 (T215000, T235295, T235656, T235217, T235295, T236846, T237556, T235231) [production]
19:03 <arlolra@deploy1001> Finished deploy [parsoid/deploy@f516018]: Updating Parsoid to 6a0a708 (duration: 10m 09s) [production]
18:58 <mholloway-shell@deploy1001> Synchronized php-1.35.0-wmf.5/extensions/MachineVision: Final fixes and tweaks for testing (duration: 00m 53s) [production]
18:53 <arlolra@deploy1001> Started deploy [parsoid/deploy@f516018]: Updating Parsoid to 6a0a708 [production]
18:39 <ejegg> re-enabled Omnimail and contact de-duplication jobs [production]
18:20 <Urbanecm> Morning SWAT done [production]
18:18 <Urbanecm> Deploy security patch for T237887 [production]
18:15 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 130ef87: Add right "abusefilter-log-private" to usergroup "rollbacker" at ptwiki (T237830) (duration: 00m 53s) [production]
18:08 <XioNoX> push pfw change to add recdns anycast IP [production]
17:33 <XioNoX> update fasw-c-eqiad to match current standard (ntp/users/rootpw/lldp) [production]
17:22 <XioNoX> update fasw-c-codfw to match current standard (ntp/users/rootpw/lldp) [production]
17:03 <ema> pool cp3052 with ATS backend T238085 [production]
17:03 <ema> pool cp3052 with ATS backend T227432 [production]
16:53 <bblack> cpNNNN (all cache nodes) - cumin manual removal of globalsign-2018 remnants (key, cert, ocsp config, ocsp output) [production]
16:42 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:40 <ema@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:28 <XioNoX> setup bgp session from cr2-codfw to multihop RIS collector - T106056 [production]
16:21 <XioNoX> reboot scs-c1-eqiad.mgmt.eqiad.wmnet - T238036 [production]
16:09 <ema> depool cp3052 and observe performance impact T238085 before reimaging as text_ats T227432 [production]
15:49 <marostegui> Deploy schema change on db1102:3315 T233135 T234066 [production]
15:45 <mholloway-shell@deploy1001> Synchronized php-1.35.0-wmf.5/extensions/MachineVision: Fixes and tweaks for initial rollout (duration: 00m 53s) [production]
15:41 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1097:3315 for a schema change T233135 T234066', diff saved to https://phabricator.wikimedia.org/P9600 and previous config saved to /var/cache/conftool/dbconfig/20191112-154127-marostegui.json [production]
15:24 <otto@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=schema [production]
14:46 <bblack> cpNNNN (all caches): remove stale outputs from transient ocsp failures ( /var/cache/ocsp/update-ocsp-*.tmp ) [production]
14:41 <ema> cp4022: trafficserver (8.0.5-1wm10) and fifo-log-demux (0.6) upgrade and restart [production]
14:38 <ema@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp4021.ulsfo.wmnet,service=nginx [production]
14:35 <ema> cp4021: ats-tls-restart to see if https://gerrit.wikimedia.org/r/550475 fixed the script [production]
14:16 <Jeff_Green> authdns-update to deploy fundraising-read.wmnet service cname adjustment [production]
14:01 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Revert "Set all of wikidata for write both for term store" (duration: 00m 52s) [production]
12:57 <godog> refresh kibana field list [production]
12:46 <gehel> repool wdqs1004 [production]
12:37 <Amir1> ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildPropertyTerms.php --wiki=wikidatawiki --batch-size 100 (T237984) [production]
12:19 <onimisionipe> restarting blazegraph on wdqs1005 [production]
12:11 <effie> Reimage mwdebug1002 - T214734 [production]
11:47 <Amir1> EU SWAT is done [production]
11:47 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.4/extensions/Wikibase: Wikibase term store error reduction, [[gerrit:550441|Do not catch DBError in ReplicaMasterAwareRecordIdsAcquirer.]] (T236466) (duration: 00m 56s) [production]
11:44 <effie> Upgrade wtp* to 7.2.24-1 with elegance and restart php-fpm - T237239 [production]
11:20 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:548584|Set all of wikidata for write both for term store (T225055)]] (duration: 00m 52s) [production]
11:05 <urbanecm@deploy1001> Synchronized wmf-config/CommonSettings.php: SECURITY: Dont allow Wikimedia sysops to see who had 2FA disabled (duration: 00m 53s) [production]
10:44 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1083', diff saved to https://phabricator.wikimedia.org/P9599 and previous config saved to /var/cache/conftool/dbconfig/20191112-104400-marostegui.json [production]
10:36 <marostegui@cumin1001> dbctl commit (dc=all): 'More traffic to db1083', diff saved to https://phabricator.wikimedia.org/P9598 and previous config saved to /var/cache/conftool/dbconfig/20191112-103641-marostegui.json [production]
10:35 <onimisionipe> resetting cronfile on wdqs hosts [production]
10:33 <marostegui> Drop labtestwiki database from m5 master db1133 - T236010 [production]
10:30 <marostegui> Deploy schema change on dbstore1003:3315 [production]
10:06 <ema> repool cp3065, nothing interesting in kern.log and SEL T238032 [production]
09:52 <marostegui@cumin1001> dbctl commit (dc=all): 'More traffic to db1083', diff saved to https://phabricator.wikimedia.org/P9596 and previous config saved to /var/cache/conftool/dbconfig/20191112-095221-marostegui.json [production]
09:42 <marostegui> Remove privileges for labtestwiki on m5 - T236010 [production]
09:27 <gehel> restarting blazegraph on wdqs1004 [production]
09:17 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1083', diff saved to https://phabricator.wikimedia.org/P9595 and previous config saved to /var/cache/conftool/dbconfig/20191112-091706-marostegui.json [production]