|
2014-01-27
§
|
| 18:37 |
<ori> |
dmesg on cp1054 [16167206.414548] XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250) |
[production] |
| 18:35 |
<ori> |
varnishd saturating cpu on cp1054 |
[production] |
| 17:26 |
<csteipp> |
synchronized php-1.23wmf10/extensions/TimedMediaHandler 'bug 56699 refix' |
[production] |
| 13:35 |
<springle> |
xtrabackup clone db1022 to db1006 |
[production] |
| 13:26 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool db1006' |
[production] |
| 10:44 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool db1015, warm up' |
[production] |
| 07:58 |
<springle> |
xtrabackup clone db1022 to db1015 |
[production] |
| 07:48 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool db1015' |
[production] |
| 07:00 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool db1023' |
[production] |
| 04:10 |
<springle> |
xtrabackup clone db1022 to db1023 |
[production] |
| 03:16 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool db1010, depool db1023' |
[production] |
| 03:09 |
<andrewbogott> |
I'm about to do a bunch of experimental bullshit in the production puppet repo, because I can't test network topology in labs. |
[production] |
| 02:44 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at 2014-01-27 02:44:23+00:00 |
[production] |
| 02:22 |
<LocalisationUpdate> |
completed (1.23wmf11) at 2014-01-27 02:22:27+00:00 |
[production] |
| 02:12 |
<LocalisationUpdate> |
completed (1.23wmf10) at 2014-01-27 02:12:03+00:00 |
[production] |
|
2014-01-25
§
|
| 07:47 |
<ori> |
auth-dns update for I0a3bf9967: Add carbon-relay & statsd service aliases |
[production] |
| 03:58 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 03:50 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Set wgVariantArticlePath for zhwikivoyage' |
[production] |
| 03:49 |
<reedy> |
updated /a/common to {{Gerrit|If943ca6e3}}: repool db1018 |
[production] |
| 02:45 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at 2014-01-25 02:45:51+00:00 |
[production] |
| 02:26 |
<LocalisationUpdate> |
completed (1.23wmf11) at 2014-01-25 02:26:16+00:00 |
[production] |
| 02:14 |
<LocalisationUpdate> |
completed (1.23wmf10) at 2014-01-25 02:14:21+00:00 |
[production] |
| 00:43 |
<awight> |
payments updated from ad67f1ca8965bb9956f371cb0fd51b503fc73764 to b458c8cd0facd28445ee9e62ef6bc0bcf31524a5 |
[production] |
| 00:43 |
<awight> |
legacy-listener updated from 1ebcfa83671b2caade50948f88c2be5cc9798692 to 5e18193c5bb703e44c5cda22009ca97c295973e3 |
[production] |
| 00:42 |
<awight> |
SmashPig updated from 3a2a746f76fffdf5fffe58ebdd9ea13c04fe7f40 to d0d59d596996f577e62f5a7186f2a2ae64098cda |
[production] |
| 00:42 |
<awight> |
tools updated from 7411ecf8745bfd93ffd25b4c97a82772d8634fbd to d67b054b584a2e0ae75744715de8f67101ae9be5 |
[production] |
|
2014-01-24
§
|
| 21:52 |
<RobH> |
old outdated (had already beed revoked) star.wikimedia.org cert removed from fenari |
[production] |
| 21:22 |
<RobH> |
confirmed no services using old star.wikimedia.org cert on zirconium, shredded cert |
[production] |
| 21:07 |
<RobH> |
removing wildcard cert from formey, which appears to be a now defunct test host per rt6134. |
[production] |
| 07:18 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool db1018' |
[production] |
| 03:02 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at 2014-01-24 03:02:40+00:00 |
[production] |
| 02:34 |
<LocalisationUpdate> |
completed (1.23wmf11) at 2014-01-24 02:34:30+00:00 |
[production] |
| 02:31 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool db1018 for schema changes' |
[production] |
| 02:26 |
<springle> |
xtrabackup clone db1022 to db1010 |
[production] |
| 02:18 |
<LocalisationUpdate> |
completed (1.23wmf10) at 2014-01-24 02:18:35+00:00 |
[production] |
| 02:08 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool db1010 for schema changes' |
[production] |
|
2014-01-23
§
|
| 23:39 |
<hashar> |
Zuul fixed up by killing a stuck job. That somehow prevented Zuul from triggering more jobs :/ |
[production] |
| 23:33 |
<hashar> |
killed job https://integration.wikimedia.org/ci/job/pywikibot-core-tests/412/ |
[production] |
| 23:32 |
<hashar> |
jenkins / zuul broken somehow with jobs triggered but not being run in jenkins. Looking |
[production] |
| 22:20 |
<paravoid> |
testing ulsfo redundancy, ignore possible ulsfo alerts |
[production] |
| 20:45 |
<paravoid> |
setting new ulsfo-eqiad transport |
[production] |
| 20:45 |
<paravoid> |
setting up new transit for ulsfo |
[production] |
| 20:20 |
<RobH> |
further tweaks on gallium, had to fully disable all 443 use, should all be ok now |
[production] |
| 19:51 |
<RobH> |
gallium has had a few patchsets applied recently for removing https support on it, apache restarted, etc. |
[production] |
| 18:30 |
<hashar> |
Gerrit: granted jenkins-mwext-sync user the ability to CR+2, V+2 and submit changes on mediawiki/extensions.git repository |
[production] |
| 18:08 |
<cmjohnson1> |
lutetium replacing failing disk at slot 8 |
[production] |