2751-2800 of 10000 results (34ms)
2016-06-09 §
08:38 <moritzm> rolling restart of app server canaries for libtasn security update [production]
07:22 <moritzm> removed /var/log/logstash/logstash.1 on logstash1001, logspam (similar to the what is described in https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/144) depleted the space on the root partition [production]
02:55 <l10nupdate@tin> ResourceLoader cache refresh completed at Thu Jun 9 02:55:20 UTC 2016 (duration 6m 19s) [production]
02:53 <mutante> ms-be2012 ran out of disk due to huge syslog, deleted log, restarted rsyslogd [production]
02:49 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.5) (duration: 11m 02s) [production]
02:26 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.4) (duration: 11m 00s) [production]
00:16 <ejegg> updated payments from de11bfe2273d0bcaa0e713389b2d91e8b3567a1d to c6ea2ca699aa78a90f64c77d860f69421e9a7782 [production]
00:15 <mutante> restarted the log bot [production]
00:11 <mutante> restarted the log bot [production]
00:06 <mutante> meta T46791 [production]
00:03 <twentyafterfour> Preparing to deploy phabricator update. Tagged release/2016-06-08/1 [production]
2016-06-08 §
23:17 <maxsem@tin> Synchronized php-1.28.0-wmf.5/extensions/WikimediaEvents/: https://gerrit.wikimedia.org/r/#/c/293439/ (duration: 00m 23s) [production]
23:15 <maxsem@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/293438 (duration: 00m 25s) [production]
23:07 <maxsem@tin> Synchronized php-1.28.0-wmf.4/extensions/LiquidThreads/: https://gerrit.wikimedia.org/r/#/c/293247/ (duration: 00m 26s) [production]
23:05 <maxsem@tin> Synchronized php-1.28.0-wmf.5/extensions/LiquidThreads/: https://gerrit.wikimedia.org/r/#/c/293247/ (duration: 00m 26s) [production]
22:51 <hoo> Re-started dumpwikidatattl on snapshot1003 [production]
22:44 <tgr@tin> Synchronized wmf-config/InitialiseSettings.php: enable AuthManager on group1 for reals T135504 (duration: 00m 25s) [production]
22:26 <tgr@tin> Synchronized wmf-config/InitialiseSettings.php: enable AuthManager on group1 T135504 (duration: 00m 23s) [production]
22:20 <tgr@tin> Synchronized php-1.28.0-wmf.5/extensions/OpenStackManager/: backport [[gerrit:293130]] for AuthManager deploy T135504 (duration: 00m 28s) [production]
22:05 <ottomata> starting kafka broker on kafka1012 after swapping disk and copying data directory [production]
22:01 <krinkle@tin> Synchronized wmf-config/CommonSettings.php: Bump wgResourceLoaderStorageVersion (T134368) (duration: 00m 28s) [production]
21:16 <mobrovac> change-prop reverting back to 96337cd540a2 [production]
21:11 <thcipriani@tin> rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.28.0-wmf.5 [production]
21:04 <thcipriani@tin> Synchronized php-1.28.0-wmf.5/includes/specials/SpecialSearch.php: [[gerrit:293422|Add a visual clear to Special:Search input box and profile-tabs]] (duration: 00m 23s) [production]
20:57 <thcipriani@tin> Synchronized php-1.28.0-wmf.5/extensions/Renameuser/RenameuserSQL.php: [[gerrit:293383|Use master DB when touching the user to signal rename end]] (duration: 00m 22s) [production]
20:50 <thcipriani@tin> Synchronized php-1.28.0-wmf.5/includes/libs/objectcache/WANObjectCache.php: [[gerrit:293363|Avoid getWithSetCallback() warnings on unversioned key migration]] (duration: 00m 24s) [production]
20:21 <thcipriani@tin> Synchronized php-1.28.0-wmf.5/extensions/Kartographer/styles/kartographer.less: [[gerrit:293360|Fixed <maplink> autostyling]] (duration: 00m 26s) [production]
20:18 <thcipriani@tin> Synchronized php-1.28.0-wmf.4/extensions/Kartographer: late SWAT: [[gerrit:293251|Fix color extraction]] (duration: 00m 36s) [production]
19:28 <mobrovac> change-prop deploying 08a1b1d [production]
19:27 <hashar> gallium enabling puppet again now that zuul/jenkins are back [production]
19:18 <hashar> Bringing back Jenkins and Zuul on gallium T137265 [production]
18:59 <ori@palladium> conftool action : set/pooled=yes; selector: name=scb1002.eqiad.wmnet [production]
18:57 <yurik> switched kartotherian to scap3, deployed, restarted [production]
18:20 <gehel> switching maps to scap3 deployment [production]
16:50 <jynus> cloning /var/lib/jenkins from db1085 to contint1001 [production]
16:46 <ottomata> stopping kafka broker and puppet on kafka1012 to replace sdf [production]
16:37 <ottomata> powercycling scb1002 [production]
16:36 <hashar> Disabled puppet on contint1001 to prevent it from bringing back Jenkins [production]
16:32 <otto@palladium> conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=mathoid']) [production]
16:32 <otto@palladium> conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=ores']) [production]
16:32 <otto@palladium> conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=mobileapps']) [production]
16:32 <otto@palladium> conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=cxserver']) [production]
16:32 <otto@palladium> conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=citoid']) [production]
16:32 <otto@palladium> conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=graphoid']) [production]
16:24 <ottomata> restarting hadoop-yarn-resourcemanager on analytics1002 to make analytics1001 active [production]
16:07 <mobrovac> scb1002 enabling back puppet [production]
16:02 <elukey> temporary set a 10TB upperbound to the Kafka webrequest_text topic to free space (T136690) [production]
15:43 <ottomata> restarting zk in codfw and eqiad 1 by 1 to apply maxClientCnxns=1024 [production]
15:12 <ottomata> restarting zookeeper 1 by 1 in eqiad [production]
15:03 <_joe_> contint1001: systemctl mask zuul,zuul-merger [production]