2201-2250 of 10000 results (26ms)
2020-11-05 §
08:43 <marostegui@cumin1001> dbctl commit (dc=all): 'es1029 (re)pooling @ 10%: Slowly pool es1029 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13211 and previous config saved to /var/cache/conftool/dbconfig/20201105-084323-root.json [production]
08:42 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1090:3312', diff saved to https://phabricator.wikimedia.org/P13210 and previous config saved to /var/cache/conftool/dbconfig/20201105-084250-marostegui.json [production]
08:33 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1090:3312', diff saved to https://phabricator.wikimedia.org/P13209 and previous config saved to /var/cache/conftool/dbconfig/20201105-083304-marostegui.json [production]
08:31 <marostegui@cumin1001> dbctl commit (dc=all): 'db1129 (re)pooling @ 100%: After regenerating table stats', diff saved to https://phabricator.wikimedia.org/P13208 and previous config saved to /var/cache/conftool/dbconfig/20201105-083142-root.json [production]
08:16 <marostegui@cumin1001> dbctl commit (dc=all): 'db1129 (re)pooling @ 75%: After regenerating table stats', diff saved to https://phabricator.wikimedia.org/P13207 and previous config saved to /var/cache/conftool/dbconfig/20201105-081638-root.json [production]
08:01 <marostegui@cumin1001> dbctl commit (dc=all): 'db1129 (re)pooling @ 50%: After regenerating table stats', diff saved to https://phabricator.wikimedia.org/P13206 and previous config saved to /var/cache/conftool/dbconfig/20201105-080135-root.json [production]
07:56 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool es1031 on es3 with minimium weight after being cloned from es1017 T261717', diff saved to https://phabricator.wikimedia.org/P13205 and previous config saved to /var/cache/conftool/dbconfig/20201105-075625-marostegui.json [production]
07:55 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool es1030 on es2 with minimium weight after being cloned from es1013 T261717', diff saved to https://phabricator.wikimedia.org/P13204 and previous config saved to /var/cache/conftool/dbconfig/20201105-075507-marostegui.json [production]
07:53 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool es1029 on es1 with minimium weight after being cloned from es1016 T261717', diff saved to https://phabricator.wikimedia.org/P13203 and previous config saved to /var/cache/conftool/dbconfig/20201105-075358-marostegui.json [production]
07:46 <marostegui@cumin1001> dbctl commit (dc=all): 'db1129 (re)pooling @ 25%: After regenerating table stats', diff saved to https://phabricator.wikimedia.org/P13202 and previous config saved to /var/cache/conftool/dbconfig/20201105-074631-root.json [production]
07:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1129 T267216', diff saved to https://phabricator.wikimedia.org/P13201 and previous config saved to /var/cache/conftool/dbconfig/20201105-072352-marostegui.json [production]
07:10 <marostegui@cumin1001> dbctl commit (dc=all): 'es1016 (re)pooling @ 100%: After cloning es1029 T261717', diff saved to https://phabricator.wikimedia.org/P13200 and previous config saved to /var/cache/conftool/dbconfig/20201105-071017-root.json [production]
07:06 <marostegui@cumin1001> dbctl commit (dc=all): 'es1013 (re)pooling @ 100%: After cloning es1030 T261717', diff saved to https://phabricator.wikimedia.org/P13199 and previous config saved to /var/cache/conftool/dbconfig/20201105-070616-root.json [production]
07:06 <marostegui@cumin1001> dbctl commit (dc=all): 'es1017 (re)pooling @ 100%: After cloning es1031 T261717', diff saved to https://phabricator.wikimedia.org/P13198 and previous config saved to /var/cache/conftool/dbconfig/20201105-070610-root.json [production]
06:55 <marostegui@cumin1001> dbctl commit (dc=all): 'es1016 (re)pooling @ 75%: After cloning es1029 T261717', diff saved to https://phabricator.wikimedia.org/P13197 and previous config saved to /var/cache/conftool/dbconfig/20201105-065514-root.json [production]
06:51 <marostegui@cumin1001> dbctl commit (dc=all): 'es1013 (re)pooling @ 75%: After cloning es1030 T261717', diff saved to https://phabricator.wikimedia.org/P13196 and previous config saved to /var/cache/conftool/dbconfig/20201105-065113-root.json [production]
06:51 <marostegui@cumin1001> dbctl commit (dc=all): 'es1017 (re)pooling @ 75%: After cloning es1031 T261717', diff saved to https://phabricator.wikimedia.org/P13195 and previous config saved to /var/cache/conftool/dbconfig/20201105-065107-root.json [production]
06:40 <marostegui@cumin1001> dbctl commit (dc=all): 'es1016 (re)pooling @ 50%: After cloning es1029 T261717', diff saved to https://phabricator.wikimedia.org/P13193 and previous config saved to /var/cache/conftool/dbconfig/20201105-064010-root.json [production]
06:36 <marostegui@cumin1001> dbctl commit (dc=all): 'es1013 (re)pooling @ 50%: After cloning es1030 T261717', diff saved to https://phabricator.wikimedia.org/P13192 and previous config saved to /var/cache/conftool/dbconfig/20201105-063610-root.json [production]
06:36 <marostegui@cumin1001> dbctl commit (dc=all): 'es1017 (re)pooling @ 50%: After cloning es1031 T261717', diff saved to https://phabricator.wikimedia.org/P13191 and previous config saved to /var/cache/conftool/dbconfig/20201105-063603-root.json [production]
06:34 <elukey> truncate application_1601916545561_129457's taskmanager.log (~600G) on an-worker1113 due to partition 'e' full [production]
06:30 <elukey> truncate application_1601916545561_129457's taskmanager.log (~600G) on an-worker1113 due to partition 'e' full [analytics]
06:25 <marostegui@cumin1001> dbctl commit (dc=all): 'es1016 (re)pooling @ 25%: After cloning es1029 T261717', diff saved to https://phabricator.wikimedia.org/P13190 and previous config saved to /var/cache/conftool/dbconfig/20201105-062507-root.json [production]
06:24 <marostegui@cumin1001> dbctl commit (dc=all): 'es1013 (re)pooling @ 25%: After cloning es1027 T261717', diff saved to https://phabricator.wikimedia.org/P13189 and previous config saved to /var/cache/conftool/dbconfig/20201105-062454-root.json [production]
06:24 <marostegui@cumin1001> dbctl commit (dc=all): 'es1017 (re)pooling @ 25%: After cloning es1028 T261717', diff saved to https://phabricator.wikimedia.org/P13188 and previous config saved to /var/cache/conftool/dbconfig/20201105-062446-root.json [production]
02:05 <milimetric> deployed refinery pointing to refinery-source v0.0.138 [analytics]
01:57 <milimetric@deploy1001> Finished deploy [analytics/refinery@6913407] (thin): Regular analytics weekly train THIN [analytics/refinery@6913407] (duration: 00m 08s) [production]
01:56 <milimetric@deploy1001> Started deploy [analytics/refinery@6913407] (thin): Regular analytics weekly train THIN [analytics/refinery@6913407] [production]
01:56 <milimetric@deploy1001> Finished deploy [analytics/refinery@6913407]: Regular analytics weekly train [analytics/refinery@6913407] (duration: 08m 34s) [production]
01:47 <milimetric@deploy1001> Started deploy [analytics/refinery@6913407]: Regular analytics weekly train [analytics/refinery@6913407] [production]
2020-11-04 §
22:04 <Krinkle> restarting cvn-app servers [cvn]
21:33 <hashar> devtools upgraded gerrit da40615f726..9275b308ffd [releng]
20:36 <Urbanecm> Late B&C Morning window completed, deployment host is clear [production]
20:36 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: ee0ba541fa55f6707276fdc5bd3f032cb9be3e60: Disable the search in header A/B test (T265333) (duration: 01m 06s) [production]
20:33 <ejegg> updated payments-wiki from 1ad4ba9639 to 388490e86d [production]
20:33 <hashar> devtools: fixed java version on gerrit-prod-1001.devtools.eqiad.wmflabs by setting java::profile hieradata variable at https://horizon.wikimedia.org/project/instances/48c5ff1c-2885-410d-beeb-d5a57a0a91c7/ [releng]
20:31 <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Migrate NewcomerTask event stream to EventGate on testwiki - T259163 (duration: 01m 07s) [production]
20:17 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 82579bf9d71bd3c9d97da0132ce8d92a8863da5b: Enable wgImagePreconnect on remaining wikis (T123582) (duration: 01m 06s) [production]
20:12 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: d2a57725f8f6fdaa3f40c834e84b43a0260077f2: Enable DiscussionTools as a beta feature on almost all Wikipedias (T266303) (duration: 01m 07s) [production]
20:08 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: fb5c03262c20b5e99b3c2f6e91abb024f12da1f5: Enable wgCheckUserLogLogins at all wikis but loginwiki (T253802) (duration: 01m 08s) [production]
19:59 <hashar> operations/software/gerrit : pushed upstream git tag, synced the master branch # T262241 [releng]
19:59 <brennen@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.16 (duration: 62m 44s) [production]
18:57 <brennen@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.16 [production]
18:52 <brennen@deploy1001> Pruned MediaWiki: 1.36.0-wmf.10 (duration: 27m 38s) [production]
18:51 <Urbanecm> Strip 2FA for Mark83 at SUL (T267257) [production]
18:50 <dpifke> Re-cherry-picked https://gerrit.wikimedia.org/r/c/operations/puppet/+/639216 in beta. [releng]
18:20 <elukey> restart memcached on mc1036 to pick up new settings (see https://gerrit.wikimedia.org/r/639099) [production]
18:15 <hknust> holger@mwmaint1002 END - Run updateRestrictions.php [production]
18:05 <dpifke> Cherry-picking https://gerrit.wikimedia.org/r/c/analytics/statsv/+/639223 in beta. [releng]
17:50 <dpifke> Reverting previous cherry-pick for now; requires minor code change first. [releng]