|
2016-06-09
§
|
| 08:54 |
<moritzm> |
installing libtasn security updates |
[production] |
| 08:38 |
<moritzm> |
rolling restart of app server canaries for libtasn security update |
[production] |
| 07:22 |
<moritzm> |
removed /var/log/logstash/logstash.1 on logstash1001, logspam (similar to the what is described in https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/144) depleted the space on the root partition |
[production] |
| 02:55 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Jun 9 02:55:20 UTC 2016 (duration 6m 19s) |
[production] |
| 02:53 |
<mutante> |
ms-be2012 ran out of disk due to huge syslog, deleted log, restarted rsyslogd |
[production] |
| 02:49 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.5) (duration: 11m 02s) |
[production] |
| 02:26 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.4) (duration: 11m 00s) |
[production] |
| 00:16 |
<ejegg> |
updated payments from de11bfe2273d0bcaa0e713389b2d91e8b3567a1d to c6ea2ca699aa78a90f64c77d860f69421e9a7782 |
[production] |
| 00:15 |
<mutante> |
restarted the log bot |
[production] |
| 00:11 |
<mutante> |
restarted the log bot |
[production] |
| 00:06 |
<mutante> |
meta T46791 |
[production] |
| 00:03 |
<twentyafterfour> |
Preparing to deploy phabricator update. Tagged release/2016-06-08/1 |
[production] |
|
2016-06-08
§
|
| 23:17 |
<maxsem@tin> |
Synchronized php-1.28.0-wmf.5/extensions/WikimediaEvents/: https://gerrit.wikimedia.org/r/#/c/293439/ (duration: 00m 23s) |
[production] |
| 23:15 |
<maxsem@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/293438 (duration: 00m 25s) |
[production] |
| 23:07 |
<maxsem@tin> |
Synchronized php-1.28.0-wmf.4/extensions/LiquidThreads/: https://gerrit.wikimedia.org/r/#/c/293247/ (duration: 00m 26s) |
[production] |
| 23:05 |
<maxsem@tin> |
Synchronized php-1.28.0-wmf.5/extensions/LiquidThreads/: https://gerrit.wikimedia.org/r/#/c/293247/ (duration: 00m 26s) |
[production] |
| 22:51 |
<hoo> |
Re-started dumpwikidatattl on snapshot1003 |
[production] |
| 22:44 |
<tgr@tin> |
Synchronized wmf-config/InitialiseSettings.php: enable AuthManager on group1 for reals T135504 (duration: 00m 25s) |
[production] |
| 22:26 |
<tgr@tin> |
Synchronized wmf-config/InitialiseSettings.php: enable AuthManager on group1 T135504 (duration: 00m 23s) |
[production] |
| 22:20 |
<tgr@tin> |
Synchronized php-1.28.0-wmf.5/extensions/OpenStackManager/: backport [[gerrit:293130]] for AuthManager deploy T135504 (duration: 00m 28s) |
[production] |
| 22:05 |
<ottomata> |
starting kafka broker on kafka1012 after swapping disk and copying data directory |
[production] |
| 22:01 |
<krinkle@tin> |
Synchronized wmf-config/CommonSettings.php: Bump wgResourceLoaderStorageVersion (T134368) (duration: 00m 28s) |
[production] |
| 21:16 |
<mobrovac> |
change-prop reverting back to 96337cd540a2 |
[production] |
| 21:11 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.28.0-wmf.5 |
[production] |
| 21:04 |
<thcipriani@tin> |
Synchronized php-1.28.0-wmf.5/includes/specials/SpecialSearch.php: [[gerrit:293422|Add a visual clear to Special:Search input box and profile-tabs]] (duration: 00m 23s) |
[production] |
| 20:57 |
<thcipriani@tin> |
Synchronized php-1.28.0-wmf.5/extensions/Renameuser/RenameuserSQL.php: [[gerrit:293383|Use master DB when touching the user to signal rename end]] (duration: 00m 22s) |
[production] |
| 20:50 |
<thcipriani@tin> |
Synchronized php-1.28.0-wmf.5/includes/libs/objectcache/WANObjectCache.php: [[gerrit:293363|Avoid getWithSetCallback() warnings on unversioned key migration]] (duration: 00m 24s) |
[production] |
| 20:21 |
<thcipriani@tin> |
Synchronized php-1.28.0-wmf.5/extensions/Kartographer/styles/kartographer.less: [[gerrit:293360|Fixed <maplink> autostyling]] (duration: 00m 26s) |
[production] |
| 20:18 |
<thcipriani@tin> |
Synchronized php-1.28.0-wmf.4/extensions/Kartographer: late SWAT: [[gerrit:293251|Fix color extraction]] (duration: 00m 36s) |
[production] |
| 19:28 |
<mobrovac> |
change-prop deploying 08a1b1d |
[production] |
| 19:27 |
<hashar> |
gallium enabling puppet again now that zuul/jenkins are back |
[production] |
| 19:18 |
<hashar> |
Bringing back Jenkins and Zuul on gallium T137265 |
[production] |
| 18:59 |
<ori@palladium> |
conftool action : set/pooled=yes; selector: name=scb1002.eqiad.wmnet |
[production] |
| 18:57 |
<yurik> |
switched kartotherian to scap3, deployed, restarted |
[production] |
| 18:20 |
<gehel> |
switching maps to scap3 deployment |
[production] |
| 16:50 |
<jynus> |
cloning /var/lib/jenkins from db1085 to contint1001 |
[production] |
| 16:46 |
<ottomata> |
stopping kafka broker and puppet on kafka1012 to replace sdf |
[production] |
| 16:37 |
<ottomata> |
powercycling scb1002 |
[production] |
| 16:36 |
<hashar> |
Disabled puppet on contint1001 to prevent it from bringing back Jenkins |
[production] |
| 16:32 |
<otto@palladium> |
conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=mathoid']) |
[production] |
| 16:32 |
<otto@palladium> |
conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=ores']) |
[production] |
| 16:32 |
<otto@palladium> |
conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=mobileapps']) |
[production] |
| 16:32 |
<otto@palladium> |
conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=cxserver']) |
[production] |
| 16:32 |
<otto@palladium> |
conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=citoid']) |
[production] |
| 16:32 |
<otto@palladium> |
conftool action : set/pooled=no; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=graphoid']) |
[production] |
| 16:24 |
<ottomata> |
restarting hadoop-yarn-resourcemanager on analytics1002 to make analytics1001 active |
[production] |
| 16:07 |
<mobrovac> |
scb1002 enabling back puppet |
[production] |
| 16:02 |
<elukey> |
temporary set a 10TB upperbound to the Kafka webrequest_text topic to free space (T136690) |
[production] |
| 15:43 |
<ottomata> |
restarting zk in codfw and eqiad 1 by 1 to apply maxClientCnxns=1024 |
[production] |
| 15:12 |
<ottomata> |
restarting zookeeper 1 by 1 in eqiad |
[production] |