|
2016-09-23
§
|
| 22:05 |
<matt_flaschen> |
Deployed patch for T146425 |
[production] |
| 21:42 |
<ebernhardson@tin> |
Synchronized php-1.28.0-wmf.20/extensions/CirrusSearch/includes/ElasticsearchIntermediary.php: Additional logging to track down autocomplete timing regression (duration: 00m 50s) |
[production] |
| 20:52 |
<gehel> |
cleaning up leftover system unit files on wdqs1* |
[production] |
| 18:41 |
<gehel> |
killing stuck tilerator notification processes on maps1001 - T145534 |
[production] |
| 17:57 |
<mutante> |
mira restarted cron |
[production] |
| 17:53 |
<ejegg> |
updated SmashPig from 8ac116037440746eaf64b9e99e1ee962d5d33475 to 372cd4008fee3fd02ad2eae9163cf7b28d2ef7c8 |
[production] |
| 17:46 |
<thcipriani@tin> |
Synchronized README: Test sync for new mira (duration: 01m 27s) |
[production] |
| 17:43 |
<mutante> |
mira - changing UID of l10nupdate to 10002, chown'ing files (1001 -> 10002) |
[production] |
| 17:35 |
<ebernhardson@tin> |
Synchronized php-1.28.0-wmf.20/extensions/CirrusSearch/includes/ElasticsearchIntermediary.php: Add timing marks to narrow down autocomplete timing regression (duration: 00m 50s) |
[production] |
| 17:31 |
<ebernhardson@tin> |
Synchronized php-1.28.0-wmf.20/extensions/CirrusSearch/includes/CompletionSuggester.php: Add timing marks to narrow down autocomplete timing regression (duration: 18m 43s) |
[production] |
| 17:04 |
<mutante> |
stat1002 - before it was hanging and then fixed due to https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hadoop/Administration#Fixing_HDFS_mount_at_.2Fmnt.2Fhdfs |
[production] |
| 17:03 |
<mutante> |
stat1002 - starting nagios-nrpe-server |
[production] |
| 14:55 |
<jynus> |
deployed dns update (removing db1010) T129395 |
[production] |
| 12:20 |
<moritzm> |
rearmed keyholder on mira |
[production] |
| 12:03 |
<_joe_> |
rolling restart of mw1280-90, high cpu usage due to memory leaks. |
[production] |
| 10:16 |
<moritzm> |
reimaging mira to jessie (again, previously installer config still pointed to trusty) |
[production] |
| 10:05 |
<Amir1> |
ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/PopulateDatabase.php --wiki=wikidatawiki (T146461) and for 'trwiki', 'plwiki', 'fawiki', 'nlwiki', 'ruwiki', 'ptwiki' |
[production] |
| 10:00 |
<Amir1> |
ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/PopulateDatabase.php --wiki=enwiki |
[production] |
| 09:58 |
<hashar@tin> |
Synchronized php-1.28.0-wmf.20/extensions/ORES/includes/Cache.php: No int typehinting (causes jobs to crash) T146461 (duration: 00m 42s) |
[production] |
| 09:58 |
<moritzm> |
rearmed keyholder on mira |
[production] |
| 09:48 |
<jynus> |
disabling alerts and shutting down db1010 in preparation for decommissioning T129395 |
[production] |
| 09:08 |
<moritzm> |
reimaging mira to jessie |
[production] |
| 09:06 |
<elukey> |
reboot eventlog2001.codfw.wmnet for kernel upgrades |
[production] |
| 08:52 |
<elukey> |
upgrading varnishkafka to 1.0.12-1 in cache:misc |
[production] |
| 08:44 |
<ema> |
depooled nginx restart on cp4003 and cp1045 for libssl upgrade |
[production] |
| 08:30 |
<elukey> |
upgrading varnishkafka to 1.0.12-1 in cache:maps |
[production] |
| 07:33 |
<elukey> |
executed 'find /var/log/hhvm/ -type f -user root -exec chown www-data:www-data {} \;' for all the api and appservers to remove/prevent cronspam (root:adm files also related to new reimaged hosts, Rsyslog needs to be configured before hhvm) - T132324 |
[production] |
| 07:02 |
<moritzm> |
rebooting francium for kernel security update |
[production] |
| 04:03 |
<aaron@tin> |
Synchronized php-1.28.0-wmf.20/includes/deferred: 5af1b93db1bb3d14844c55e4e3ed17fe963de551 (duration: 00m 48s) |
[production] |
| 04:02 |
<aaron@tin> |
Synchronized php-1.28.0-wmf.20/includes/libs/rdbms: 5af1b93db1bb3d14844c55e4e3ed17fe963de551 (duration: 00m 51s) |
[production] |
| 02:46 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Fri Sep 23 02:46:04 UTC 2016 (duration 6m 10s) |
[production] |
| 02:39 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.20) (duration: 17m 04s) |
[production] |
| 02:13 |
<maxsem@tin> |
Synchronized php-1.28.0-wmf.20/extensions/SecurePoll/: https://gerrit.wikimedia.org/r/#/c/312450/1 (duration: 00m 51s) |
[production] |
| 02:10 |
<mutante> |
mw1206, mw1224 - restarted hhvm and apache |
[production] |
| 01:49 |
<bblack> |
depooled mw1224 service apache2 |
[production] |
| 00:38 |
<Krenair> |
mw1224 apache stuck, not restarting for now in case someone wants to investigate later. possibly T89912? |
[production] |
| 00:17 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/312339 (duration: 00m 48s) |
[production] |
| 00:16 |
<krenair@tin> |
Synchronized wmf-config/mobile.php: https://gerrit.wikimedia.org/r/312339 (duration: 00m 47s) |
[production] |
| 00:08 |
<krenair@tin> |
Synchronized php-1.28.0-wmf.20/extensions/FlaggedRevs/business/RevisionReviewForm.php: https://gerrit.wikimedia.org/r/#/c/312423/ (duration: 00m 48s) |
[production] |
|
2016-09-22
§
|
| 23:46 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/310483 (duration: 00m 48s) |
[production] |
| 23:19 |
<krenair@tin> |
Synchronized php-1.28.0-wmf.20/resources/src/mediawiki.less/mediawiki.ui/mixins.less: https://gerrit.wikimedia.org/r/#/c/312340/ (duration: 00m 48s) |
[production] |
| 22:49 |
<aaron@tin> |
Synchronized php-1.28.0-wmf.20/includes/libs/rdbms/loadbalancer/LoadBalancer.php: a73a7ef9286275f797411646f9c5af60d4894c73 (duration: 01m 04s) |
[production] |
| 22:18 |
<mutante> |
added slaporte and zhousquared to wmf LDAP group (T146227) |
[production] |
| 21:24 |
<hasharAway> |
Nodepool is all back and operational. Reduced amount of queries to the OpenStack API by more than 10% |
[production] |
| 21:14 |
<yurik> |
deployed tilerator https://gerrit.wikimedia.org/r/#/c/312329/ |
[production] |
| 21:05 |
<hasharAway> |
stopped nodepooled and restarted it with 0.1.1-wmf5 |
[production] |
| 21:04 |
<mutante> |
upgradede nodepool to 0.1.1-wmf5 on labnodepool1001 |
[production] |
| 21:04 |
<mutante> |
upgradede nodepool to 0.1.1-wmf on labnodepool1001 |
[production] |
| 21:04 |
<krinkle@tin> |
Synchronized php-1.28.0-wmf.20/resources/src/mediawiki/mediawiki.js: T146099 (duration: 00m 48s) |
[production] |