February 14th :Our visual search engine was down during night

Our visual search engine was down in the night from Thursday February 14th to Friday February 15th, between 1:55 am and 7:58 am (UTC). Unfortunately our alarming system didn't work properly in this particular case, so that none of our engineers got alerted to fix it. As soon as we were aware of the problem, we were able to fix it within 10 minutes. 
During that period, our website and the Layar Creator were functioning normally, but users of our Layar app were not able to find results when scanning any page augmented with Layar.
We know the cause of the downtime and have already taken measures to avoid it in the future. An out-of-memory situation occurred on one of our server components and we have now installed automatic jobs that will detect such a situation and restart these components. Also we are currently changing our software to avoid such out-of-memory situations completely.
Also we are checking all our monitoring alerting systems to ensure they will wake us up at night if such a situation occurs again. Our services are supposed to run 24/7 and we are aware that customers all over the world rely on our services being available at any time of the day.
We are terribly sorry for the inconvenience this outage has caused.
Dirk Groten
CTO Layar
Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request


Powered by Zendesk