1351-1400 of 2908 results (18ms)
2018-10-09 §
12:48 <elukey> re-run all the failed projectview-hourly-coord and aqs-hourly-coord workflows (restarting them via hue) [analytics]
12:47 <elukey> re-run apis-wf-2018-10-9-8 [analytics]
10:01 <joal> Restart failed oozie jobs (webrequest, virtual-pageviews, mwh-reduced) [analytics]
07:14 <elukey> stopped all crons on analytics1003 as prep step for migration to an-coord1001 [analytics]
2018-10-08 §
16:28 <elukey> restart eventlogging on eventlog1002 for python security upgrades [analytics]
10:26 <elukey> swapped db settings from analytics1003 to an-coord1001 on both Druid clusters (restarted coordinators and overlords) [analytics]
07:35 <joal> Manually run download-project-namespace-map with proxy [analytics]
2018-10-06 §
18:10 <elukey> restart Yarn Resource Manager on an-master1002 to force an-master1001 to take the active role back (failed over due to a zk conn issue) [analytics]
2018-10-05 §
10:32 <elukey> piwik/matomo out of maintenance [analytics]
10:17 <elukey> set piwik/matomo in maintenance mode on matomo1001 [analytics]
2018-10-04 §
20:33 <mforns> Finished deployment of refinery [analytics]
19:52 <mforns> Started deployment of refinery [analytics]
19:50 <mforns> Finished deployment of refinery-source [analytics]
19:22 <mforns> Started deployment of refinery-source [analytics]
17:20 <elukey> bounce druid-brokers on druid100[4-6] after network maintenance [analytics]
2018-10-01 §
12:56 <fdans> reverting to last version of wikistats [analytics]
2018-09-27 §
06:44 <elukey> rolling restart of Druid coordinators and historicals on the Druid public cluster to pick up new Hadoop masters (one at the time, very gently) [analytics]
2018-09-26 §
20:39 <elukey> rolling restart of all the druid historicals on Druid private/analytics [analytics]
20:00 <ottomata> rolling restart of druid coordinators to hopefully pick up hadoop master config change [analytics]
17:49 <joal> Deploy AQS from scap [analytics]
08:22 <elukey> start mysql consumers on eventlog1002 after maintenance [analytics]
07:51 <elukey> stop mysql consumers on eventlog1002 as prep step for db maintenance [analytics]
2018-09-25 §
20:21 <joal> Webrequest warning for upload-2018-09-25-13 were all false positives [analytics]
17:36 <ottomata> stopping refine jobs and deploying refinery source 0.0.75 - T203804 [analytics]
12:37 <joal> Rerun webrequest-load-wf-text-2018-9-25-6 and webrequest-load-wf-text-2018-9-25-7 after SLA failure due to hadoop master swaps [analytics]
11:55 <joal> Rerun webrequest-load-wf-upload-2018-9-25-6 after failed SLA during hadoop master swap [analytics]
11:53 <joal> rerun as you prefer dcausse :) [analytics]
08:02 <joal> Killing discovery transfer job to drain cluster before master replacement (application_1536592725821_38136) [analytics]
06:24 <elukey> stop camus crons on an1003 and report updater on stat1005 as prep step for cluster shutdown [analytics]
2018-09-20 §
16:04 <joal> webrequest-load-check_sequence_statistics-wf-text-2018-9-19-20 have been checked as false-positive [analytics]
2018-09-15 §
12:22 <joal> Restart webrequest-druid-[hourly|daily] coordinators [analytics]
12:20 <joal> Kill wikidata-wdqs coordinator [analytics]
12:11 <joal> Killing and restarting webrequest-load-bundle [analytics]
12:00 <joal> Deploying refinery onto hadoop :) [analytics]
11:39 <joal> Deploying refinery with scap [analytics]
2018-09-12 §
17:34 <ottomata> deploying new version of refinery-source, and then refinery for properties based RefineMonitor job - https://phabricator.wikimedia.org/T203804 [analytics]
13:11 <ottomata> otto@deploy1001 Started deploy [eventlogging/analytics@5c6fab6]: Support loading plugins in eventlogging-processor - T203596 [analytics]
06:21 <elukey> re-run webrequest-load-wf-text-2018-9-12-4, failed due to sql exceptions/timeouts to the database [analytics]
2018-09-10 §
16:26 <ottomata> restarting eventlogging-processors to pick up blacklist of WebClientError schema for MySQL - T203814 [analytics]
12:49 <elukey> disable camus as prep step for analytics100[1-3] reboots [analytics]
07:54 <joal> Manually restarting mediawiki-reduced oozie with manual addition of missing parameter [analytics]
2018-09-07 §
18:18 <joal> Manually downoad namespaces for 2018-08 [analytics]
17:32 <joal> Manually rerun download-project-namespace-map on analytics1003 after cron's failure [analytics]
2018-09-06 §
13:03 <fdans> restarted virtualpageview_hourly coordinator [analytics]
2018-09-05 §
18:18 <ottomata> restarted eventlogging processors blacklisting CentralNoticeImpression - T203592 [analytics]
16:56 <ottomata> restarting eventlogging processors to blacklist CitationUsage - T191086 [analytics]
14:42 <elukey> deploying refinery (pageview whitelist and cron script change) [analytics]
13:40 <ottomata> reimaging thorium to debian stretch (this will cause an announced {stats,analytics}.wm.org downtime!) - T192641 [analytics]
13:21 <fdans> restarting webrequest load bundle, start time 11:00Z [analytics]
09:12 <elukey> re-run webrequest-druid-hourly-wf-2018-9-5-7 - failed due to rebooting druid1001 [analytics]