2401-2450 of 3819 results (20ms)
2018-06-27 §
13:08 <elukey> piwik upgraded to 3.2.1 on bohrium + started the db migration procedure (will last 2/3h probably) [analytics]
12:57 <elukey> set Piwik in maintenance mode as prep step for backup + upgrade [analytics]
2018-06-20 §
19:54 <ottomata> removed Kafka MirrorMaker from kafka10(12|13|14) [analytics]
2018-06-18 §
11:57 <joal> Restart oozie webrequest refine jobs [analytics]
11:19 <joal> Launch oozie webrequest refine jobs for the failing hour 2018-06-14-11 [analytics]
10:18 <joal> Deployed refiney on hdfs [analytics]
10:18 <joal> Deployed refinery with scap [analytics]
2018-06-15 §
09:00 <joal> Deleting corrupted file hdfs://analytics-hadoop/user/joal/wmf/data/raw/webrequest/webrequest_upload/hourly/2018/06/14/11/webrequest_upload.1004.10.1214791.15490650727.1528974000000._COPYING_ to prevent webrequest refine jobs from failing. No data will be lost as the correct file exist. [analytics]
2018-06-14 §
19:29 <joal> try rerunning webrequest-load-wf-upload-2018-6-14-11 [analytics]
13:14 <elukey> re-run failed webrequest-upload/text jobs (namenodes restarted) [analytics]
2018-06-11 §
13:56 <ottomata> bouncing eventlogging processes to apply kafka event time producing [analytics]
2018-06-08 §
11:45 <joal> Launching manual sqooping of revision and archive table to recover from failure [analytics]
2018-06-01 §
08:37 <joal> Restart every druid loading oozie job (except mediawiki reduced) to pick new configuration [analytics]
08:33 <joal> Restart mediawiki-history-denormalize oozie job after deploy [analytics]
08:24 <joal> Deploy refinery on HDFS [analytics]
08:08 <joal> Deploying refinery using scap [analytics]
07:53 <joal> Releasing refinery-source v0.0.65 to archiva [analytics]
07:05 <joal> Rerun virtualpageview-druid-monthly-wf-2018-5 [analytics]
2018-05-31 §
17:01 <ottomata> dropping and deleting MobileWikiAppiOS* tables and data per request from chelsyx [analytics]
10:51 <elukey> stopped Pivot on thorium [analytics]
07:27 <joal> Restart webrequest-load-bundle with default oozie_launcher_memory value (should be 2048 set by workflows) [analytics]
05:33 <elukey> re-run faied webrequest-load upload|misc jobs via Hue [analytics]
01:02 <ottomata> bouncing main-eqiad -> jumbo-eqiad mirror maker [analytics]
2018-05-30 §
17:49 <joal> Rerun webrequest-load-wf-misc-2018-5-30-16 [analytics]
13:15 <elukey> re-run webrequest-load-wf-upload-2018-5-30-11 - died after worker node reboots [analytics]
06:14 <elukey> re-run failed webrequest-load jobs [analytics]
06:11 <elukey> temporary point Turnilo to druid1002 to allow druid1001's reimage [analytics]
05:50 <elukey> restart mirror maker on kafka10[12-23] - failures to consume after rebalance [analytics]
2018-05-29 §
17:02 <elukey> re-run webrequest-load-text 29th May 2018 12:00:00 [analytics]
15:03 <joal> rerun webrequest-load-wf-upload-2018-5-29-13 [analytics]
10:30 <elukey> roll restart of druid-middlemanagers on druid* to pick up the new runtime settings (no more references to hadoop-client-cdh) [analytics]
10:04 <elukey> re-run pageview-druid-hourly-wf-2018-5-29-7 [analytics]
07:05 <elukey> re-run webrequest-load-wf-text-2018-5-29-1 [analytics]
2018-05-28 §
18:51 <elukey> rerun webrequest-load-wf-upload-2018-5-28-14 [analytics]
18:16 <elukey> restart kafka mirror maker on kafka1012->14 - failed after the last round of kafka restarts [analytics]
12:55 <elukey> re-run webrequest-load-wf-misc-2018-5-28-10 [analytics]
05:50 <elukey> re-run webrequest-load-wf-misc-2018-5-27-22, webrequest-load-wf-text-2018-5-28-2, webrequest-load-wf-upload-2018-5-28-3 [analytics]
2018-05-27 §
07:25 <joal> Rerun webrequest-load-wf-upload-2018-5-25-23 [analytics]
07:25 <joal> rerun webrequest-load-wf-misc-2018-5-26-16 and webrequest-load-wf-misc-2018-5-27-0 [analytics]
2018-05-25 §
06:53 <elukey> re-run webrequest-load-wf-upload-2018-5-24-23 and webrequest-load-wf-text-2018-5-25-4 [analytics]
2018-05-24 §
17:20 <ottomata> dropped and deleted raw and refined eventlogging tables and data for MobileWikiAppiOSUserHistory, MobileWikiAppiOSLoginAction, MobileWikiAppiOSSettingAction, MobileWikiAppiOSReadingLists, MobileWikiAppiOSSessions [analytics]
16:45 <joal> Rerun webrequest-druid-daily-wf-2018-5-23 to correct corrupted data [analytics]
08:17 <elukey> increase webrequest replication to 2 in druid analytics (via coordinator's UI) [analytics]
08:16 <joal> rerun webrequest-load-wf-misc-2018-5-24-6 [analytics]
2018-05-23 §
14:25 <ottomata> redirecting pivot -> turnilo.wikimedia.org - https://phabricator.wikimedia.org/T194427 [analytics]
07:35 <elukey> upgrading the Druid labs cluster to Debian Stretch [analytics]
06:14 <elukey> re-run webrequest-load-wf-misc-2018-5-23-2 via Hue [analytics]
2018-05-22 §
15:38 <elukey> re-run webrequest-druid-hourly-wf-2018-5-22-12 - failed due to Druid cluster upgrade in progress [analytics]
14:08 <elukey> upgrade druid on druid100[1-3] to 0.11 [analytics]
13:37 <elukey> killed banner impression data job (application_1523429574968_110796) and removed its related respawn cron on an1003 [analytics]