|
2016-02-02
ยง
|
| 22:20 |
<ori> |
restarted HHVM on mw1243. Lock-up. Backtrace in /tmp/hhvm.2897.bt |
[production] |
| 22:20 |
<demon@mira> |
Finished scap: re-sync batch of mw1101-1135,1240-1260, 2101-2150 with master (duration: 12m 51s) |
[production] |
| 22:07 |
<demon@mira> |
Started scap: re-sync batch of mw1101-1135,1240-1260, 2101-2150 with master |
[production] |
| 22:00 |
<demon@mira> |
Finished scap: re-sync batch of mw1151-mw1225, mw2174-mw2214 with master (duration: 11m 24s) |
[production] |
| 21:48 |
<demon@mira> |
Started scap: re-sync batch of mw1151-mw1225, mw2174-mw2214 with master |
[production] |
| 21:45 |
<demon@mira> |
Finished scap: re-sync batch of mw1051-1100, mw2051-2100 with master (duration: 13m 41s) |
[production] |
| 21:31 |
<demon@mira> |
Started scap: re-sync batch of mw1051-1100, mw2051-2100 with master |
[production] |
| 21:28 |
<demon@mira> |
Finished scap: re-sync batch of mw1025-1050 and mw2007-mw2050 with master (2nd try) (duration: 14m 33s) |
[production] |
| 21:27 |
<_joe_> |
depooling eqiad scap-proxies |
[production] |
| 21:13 |
<demon@mira> |
Started scap: re-sync batch of mw1025-1050 and mw2007-mw2050 with master (2nd try) |
[production] |
| 21:04 |
<demon@mira> |
scap aborted: re-sync batch of mw1025-1050 and mw2007-mw2050 with master (duration: 10m 11s) |
[production] |
| 20:54 |
<demon@mira> |
Started scap: re-sync batch of mw1025-1050 and mw2007-mw2050 with master |
[production] |
| 20:32 |
<hashar> |
mw1114-mw1119 are canary api appservers Finished syncing |
[production] |
| 20:28 |
<ori> |
restarted hhvm on mw1116 |
[production] |
| 20:17 |
<hashar> |
Running sync-common on mw1114-mw1119 (canary api appservers) |
[production] |
| 20:16 |
<ostriches> |
mira: removed untracked wmf-config/x.php testing file |
[production] |
| 20:11 |
<ori> |
Running sync-common on canary app servers (mw1017-mw1025) |
[production] |
| 19:46 |
<hashar> |
Running sync-common on mw1260 (video scaler) |
[production] |
| 19:40 |
<ori> |
Running sync-common on all jobscalers |
[production] |
| 19:35 |
<ori> |
Running sync-common on mw1259 (video scaler) and mw1153 (image scaler) too |
[production] |
| 19:29 |
<ori> |
Running sync-common on mw100[123] |
[production] |
| 18:59 |
<_joe_> |
running sync-common on mw1020 |
[production] |
| 18:54 |
<_joe_> |
repooled mw1119 |
[production] |
| 17:45 |
<hashar> |
mira /srv/mediawiki-staging git submodule update --init --recursive |
[production] |
| 17:43 |
<hashar> |
mw1119 sync-common |
[production] |
| 17:37 |
<godog> |
disable unused swift container-sync for wikibooks-ka-local-thumb wikibooks-hr-local-thumb wikibooks-km-local-thumb wikibooks-sk-local-thumb wikibooks-tr-local-thumb wikipedia-it-local-thumb.fc |
[production] |
| 17:36 |
<hashar> |
mw1119:/srv/mediawiki/wmf-config/event-schemas is empty |
[production] |
| 17:31 |
<_joe_> |
depooled mw1119, partial sync |
[production] |
| 16:59 |
<hashar> |
files were /srv/mediawiki/docroot/wikimedia.org/WikipediaMobileFirefoxOS/.git and /srv/mediawiki/docroot/wikimedia.org/WikipediaMobileFirefoxOS/js/lib/MobileFrontend/.git |
[production] |
| 16:58 |
<ostriches> |
mw1017: removed stray .git directory from WikipediaFirefoxMobileOS or w/e. It shouldn't be there anyway. sync-common is happy again on it |
[production] |
| 16:48 |
<hashar> |
tin /srv/mediawiki-staging : running git submodule update --init --recursive |
[production] |
| 16:47 |
<hashar> |
tin /srv/mediawiki-staging : running git submodule update --init |
[production] |
| 16:40 |
<hashar> |
mw1017 sync-common --verbose |
[production] |
| 16:35 |
<_joe_> |
sync-common on mw2030 and mw1161; re-enable puppet, jobrunner, jobchron on mw1161 |
[production] |
| 16:34 |
<_joe_> |
restarted puppet and rsync on both tin and mira, removed comments on the l10nupdate job on tin |
[production] |
| 16:23 |
<thcipriani@mira> |
rebuilt wikiversions.php and synchronized wikiversions files: rebuild wikiversion.php |
[production] |
| 14:57 |
<godog> |
disable swift container-sync for wikipedia-it-local-public.a7 |
[production] |
| 14:43 |
<hashar> |
tin /srv/mediawiki-staging/multiversion/checkoutMediaWiki 1.27.0-wmf.10 php-1.27.0-wmf.10 |
[production] |
| 14:43 |
<hashar> |
tin /srv/mediawiki-staging/multiversion/checkoutMediaWiki 1.27.0-wmf.9 php-1.27.0-wmf.9 |
[production] |
| 14:43 |
<hashar> |
tin /srv/mediawiki-staging/multiversion/checkoutMediaWiki 1.27.0-wmf.8 php-1.27.0-wmf.8 |
[production] |
| 14:21 |
<hashar> |
starting rebuilding /srv/mediawiki-staging from scratch on tin (not mira) |
[production] |
| 14:20 |
<hashar> |
starting rebuilding /srv/mediawiki-staging from scratch on mira |
[production] |
| 14:04 |
<bblack> |
nevermind, not looking at eeden |
[production] |
| 14:04 |
<bblack> |
looking at eeden |
[production] |
| 13:58 |
<moritzm> |
rebooting eeden for kernel update |
[production] |
| 13:09 |
<moritzm> |
rolling reboot of scb* (for kernel update) |
[production] |
| 13:01 |
<akosiaris> |
reboot dubnium for kernel upgrades |
[production] |
| 13:01 |
<akosiaris> |
reboot pollux for kernel upgrades |
[production] |
| 12:45 |
<moritzm> |
rebooting baham for kernel update |
[production] |
| 12:20 |
<_joe_> |
stopping rsync on mira too, to avoid accidental deploys |
[production] |