Posts tagged Logentries

R7-2017-18: Logentries Windows Agent uses vulnerable OpenSSL (FIXED)

Summary The Logentries Windows Agent before version 2.6.0.1 shipped with a version of OpenSSL that is susceptible to several public vulnerabilities described below. While we have no indication that any Logentries customers have been compromised due to these older versions of OpenSSL, we strongly encourage Logentries customers to update Agents deployed to Windows systems using the steps outlined under “Remediation” below. Since the previously shipped version of OpenSSL was susceptible to severa

3 min IT Ops

Network Administrator’s Guide to Surviving an Audit: During the Audit

Last time [/2017/03/31/network-administrators-guide-to-surviving-an-audit-preparation/] we talked about how to prepare for an audit. In this installment we’ll cover what to do once the audit begins. Let’s assume that you’re pretty well prepared. You’ve done your homework and know pretty much what to expect. So, everything’s good, right? Well, even though you’ve taken the time to prepare, you could be in for some surprises. The keys to surviving an IT audit are pretty simple: * Be truthful *

7 min IT Ops

Logging in a Software Defined Network

Background This blog will give an overview of Software Defined Networks (SDN), present some suggestions for logging in an SDN and finally present an overview of some research work we are doing on SDN logging. If we consider a Software Defined Network (SDN) paradigm is a racetrack, SDN controllers are race cars. Networking vendors especially those in the telecommunication area such as Deutsche Telecom, Orange, Vodafone use their own SDN controllers to manage the orchestration of their own equi

5 min IT Ops

Habits that Pay Off for Programmers

I would like to clarify something immediately with this post. Its title does not contain the number 7, nor does it talk about effectiveness. That was intentional. I have no interest in trying to piggy-back on Stephen Covey’s book title to earn clicks, which would make this post a dime a dozen. In fact, agoogle search of “good habits for programmers” [https://www.google.com/#q=good+habits+for+programmers]yields just such an appropriation, and it also yields exactly the sorts of articles and

3 min IT Ops

Legacy apps in the data center, today's apps in the cloud

Have you found that your organization is developing new applications that are cloud-based, but unable to move away from some established legacy systems? You aren’t alone. This legacy/cloud hybrid environment is far more common than you would expect. And when you look at the history behind these apps it does make sense. Any organization that has been around for more than a few years probably has some investment in at least one legacy application. Organizations that have been around longer may h

4 min IT Ops

Network Administrator’s Guide to Surviving an Audit: Preparation

Sooner or later, your organization will likely be the subject of an IT audit. But as ominous as that sounds, it doesn’t have to be something to dread. If you’re a network administrator, you’ll have a specific role in an audit. Since audits are rarely small projects, you’ll likely be working with others throughout the process. The best way to fulfill your specific role well is to be prepared for an audit before it happens. Simply put, an audit is an examination to determine if controls are suff

4 min IT Ops

Log Analysis for System Troubleshooting

Systems of all kinds create log data constantly and voluminously. In searching out the most compelling reasons to dig into and analyze such data, we compiled a list of seven reasons that usually drive such activity. In this blog post we tackle the first of those 7, which include: 1. System troubleshooting 2. Security incident response 3. Security troubleshooting 4. Performance troubleshooting 5. Understanding user behavior or activities 6. Compliance with security policies 7. Complianc

5 min IT Ops

Logging for Fun: Things You'd Never Thought to Log

I work as a consultant in the software industry. This work affords me the opportunity to see and interact with many different teams and thus to observe prevailing trends. Among these teams, the attitude toward logging tends to be one of resigned diligence. That is, many developers view application logging the way they view flossing their teeth: a necessary, dull maintenance activity that will pay dividends later. Today, however, I’d like to encourage readers to consider a different side of

8 min IT Ops

Roots and Culture: Logging and the Telephone Bill

Telephone systems were the Internet before there was an Internet. Think about it. By 1920 millions of people were exchanging data on a worldwide network using a device that connected on demand. Sounds like the Internet to me. But unlike the current day Internet, the telephone system cost money to use. Alexander Graham Bell’s investors wanted it that way. That’s why they gave him the money. Thus, people who used the telephone system had to pay for it. So going as far back as 1877, every mont

3 min IT Ops

Never type a search query again!

Introducing Visual Search In our never-ending effort to help you wrangle your infrastructure, we are constantly improving and adding new functionality to Logentries. In that spirit, today we are happy to announce Visual Search. Visual Search is truly a one of a kind feature, it makes the whole process of an investigation shorter and simpler. By automatically visualizing top trends appearing in your data, Visual Search allows you to simply click to drill in and out of datasets to identify tren

3 min Awards

Finalists in FIVE categories at the Network Computing Awards!

Ring Ring! You're in the Final! It's always nice to get a phone call letting us know that we've been shortlisted for awards – but when it's five awards, we like those calls even more! Two of our products, and our company have reached the final stages for the Network Computing Awards, and of course we'd love it if you took a moment to vote for us please. La La Land may have racked up the Oscar noms, but at the Network Computing Awards it's looking good for LE LE Land! OK, so we might not quite

6 min IT Ops

5 Rules of Pair Programming Etiquette

I like Pair Programming [https://en.wikipedia.org/wiki/Pair_programming]. I’ve been doing it episodically for about 10 years. Whenever I’ve pair programmed, at the end of a session, I’ve always walked away a better developer than when I started. However, the practice can be expensive when the pair doing the programming are not efficient. When a lot of friction exists between the two coders involved, costs can exceed double that of a single programmer trying to hash things out on his or her ow

5 min IT Ops

3 Predictions for Development in the Age of Serverless Computing

Probably the biggest change that has happened in my lifetime of programming is the transformation from creating code that is meant to run on known, tangible hardware to making code that runs on the Cloud. We’ve gone from server based computing to the serverless environment. The transformation has brought us the practice of DevOps. Also, the transformation has forced us to rethink the whole way design our code. Increasingly modern programming is about stitching together cloud based resources t

2 min IT Ops

Java 8 - Lazy argument evaluation

Overview “I will always choose a lazy person to do a difficult job. Because he will find an easy way to do it” – Bill Gates Lazy evaluation is an evaluation strategy [https://en.wikipedia.org/wiki/Evaluation_strategy] which delays the evaluation of an expression until its value is needed. The opposite of this is eager evaluation, where an expression is evaluated as soon as it is bound to a variable.[wikipedia [https://en.wikipedia.org/wiki/Lazy_evaluation]] Like most imperative programming l

3 min Haxmas

12 Days of HaXmas: The Gift of Endpoint Visibility and Log Analytics

Merry HaXmas to you! Each year we mark the 12 Days of HaXmas [/tag/haxmas/] with 12 blog posts on hacking-related topics and roundups from the year. This year, we're highlighting some of the “gifts” we want to give back to the community. And while these gifts may not come wrapped with a bow, we hope you enjoy them. Machine generated log data is probably the simplest and one of the most used data source for everyday use cases such as troubleshooting, monitoring, security investigations … the lis

4 min IT Ops

Logs To Understand User Activity and Behavior

Logging user activity is a great way to understand what users are doing, and how they are using network and computing resources. Collecting data from the standpoint of a user identity or login is a great way to correlate all kinds of information, too, including client or workstation activity, network and server access, and application usage. This provides a unique opportunity to make use of Logentries’ [https://logentries.com/centralize-log-data-automatically/?le_trial=user_activity_and_behav

6 min IT Ops

The Value of Correlation IDs

In the old days when transactional behavior happened in a single domain, in step-by-step procedures, keeping track of request/response behavior was a simple undertaking. However, today one request to a particular domain can involve a myriad of subsequent asynchronous requests from the starting domain to others. For example, you send a request to Expedia, but behind the scenes Expedia is forwarding your request as a message to a message broker. Then that message is consumed by a hotel, airline

5 min IT Ops

The Generosity of Thought: Caring and Sharing in the Open Source Community

I want to share something with you that is pretty amazing. But, before I do, allow me to provide the backstory. The Backstory I’ve been using Open Source Software (OSS) for a while now. I started with the big ones, Apache [http://apache.org/], Maven [http://maven.apache.org/], MySQL [http://www.mysql.com/], etc…. But, as time went on and my work became more specialized, I started using smaller projects. When you use the big projects such as Maven and Apache, there’s a boatload of books, video

5 min IT Ops

Solving the expression problem

If you look at any OO-based codebase of a nontrivial size, you’ll [hopefully] find well understood behavior formalized and encapsulated through the effective use of polymorphism- either via interfaces which decouple calling code from a types’ implementation, or via sub typing to share code common to multiple types. To take an example from a statically typed language like Java, let’s look at the Map interface and a few of its implementations in the standard library: A receiving method which

4 min InsightOps

Announcing InsightOps - Pioneering Endpoint Visibility and Log Analytics

Our mission at Rapid7 is to solve complex security and IT challenges with simple, innovative solutions. Late last year Logentries joined the Rapid7 family to help to drive this mission. The Logentries technology itself had been designed to reveal the power of log data to the world and had built a community of 50,000 users on the foundations of our real time, easy to use yet powerful log management and analytics engine. Today we are excited to announce InsightOps, the next generation of Logentri

5 min IT Ops

Node.js as a Proxy to Logentries.com

Logging from the client side of a web application can seem like a challenge. The web browser exposes everything to the user. There is no way to hide anything delivered to the client from prying eyes, including your log token to your Logentries [https://logentries.com/centralize-log-data-automatically/?le_trial=nodejs_as_a_proxy-logentries_blog-post_cta-create_trial&utm_campaign=nodejs_as_a_proxy&utm_source=logentries_blog&utm_medium=post_cta&utm_content=create_trial] log. There is no relia

4 min IT Ops

Goodbye to the VCR: Rewinding Down Memory Lane

The VHS tape was a thing of a magic that is alien in the world of on-demand media. It represented a promise of entertainment. A promise only realized when loading the tape into the player and pressing the play button. There was an air of excitement around every video, and you could never be entirely certain about what the video contained. I was reminded of the mystique of the VCR just this month when I read about its final demise. Funai Electric, the last major manufacturer of VCR players,

3 min IT Ops

Widely-used Android App Leaks MS Exchange Credentials

In October, Rapid7 researchers [https://community.rapid7.com/community/infosec/blog/2016/10/11/r7-2016-21-nine-folders-certificate-validation-vulnerability-cve-2016-2533] uncovered a significant vulnerability in the Nine mobile application [https://play.google.com/store/apps/details?id=com.ninefolders.hd3&hl=en] for Android. Baldly stated, this app leaks Microsoft Exchange user credentials, plus mail envelopes and attachments, mailbox synchronization data, caleandar entries and tasks to attac

3 min IT Ops

Empowering Leexportpy with new services

In previous posts [/2016/07/05/introduction-to-leexportpy/] of this series, we have gone through how leexportpy works, how it is configured and currently supported services for transforming and pushing Logentries Query API [https://docs.logentries.com/docs/query-api] data to other systems. When it comes to creating a new service for leexportpy, the core of leexportpy is quite dynamic and flexible. Placing your new service file that conforms to the requirements [https://github.com/rapid7/leexp

4 min IT Ops

Overview of 'online' algorithm using Standard Deviation example

Here at Logentries [https://logentries.com/centralize-log-data-automatically/?le_trial=online_algorithm-logentries_blog-post_cta-create_trial&utm_campaign=online_algorithm&utm_source=logentries_blog&utm_medium=post_cta&utm_content=create_trial] we are constantly adding to the options for analysing log generated data. The query language ‘LEQL’ [https://logentries.com/resources/how-to-videos/building-a-query/] has a number of statistical functions and a recent addition has been the new Standard

5 min IT Ops

Logging OwnTracks to Logentries

A previous blog [/2016/10/07/logging-mosquitto-server-logs-from-raspberry-pi-to-logentries/] showed how MQTT logs can be sent to Logentries for storage, analysis and how those logs can be to alert on potential MQTT security threats, as well as to store and visualize sensor data. This blog follows that by showing how to build a fully connected IoT system composed of the OwnTracks iOS app as an MQTT publisher, a Raspberry Pi with Mosquitto embedded as an MQTT messaging broker and Logentries as

5 min IT Ops

Logging Mosquitto Server logs (from Raspberry Pi) to Logentries

The Internet is evolving and part of this is the emerging Internet of Things (IoT). IoT allows us to use the Internet to seamlessly connect the cyberspace and real world using physical sensors at huge scale, allowing us to gather and analyze the data across many domains. It is estimated that there will be 20 billion Things connected to the Internet by 2020, generating an enormous amount of data. A previous blog post [/2016/05/17/raspberry-pi-logs-and-iot-sending-pi-log-and-sensor-data-to-loge

4 min IT Ops

Using Event and Log Data to Assure Security and Compliance

All the way back in 2007, security industry analyst and expert Dr. Larry Ponemon of the eponymous institute wrote for Network World [https://web.archive.org/web/20080820033845/http:/www.networkworld.com/news/2007/051507-data-breaches.html] that “data breaches are a pervasive problem for most organizations in the United States today.” Nobody is prepared to argue that this situation has improved much, if at all, 9 years later in 2016. However, those organizations that exert themselves to make t

2 min Nexpose

UNITED 2016: Want to share your experience?

Key trends. Expert advice. The latest techniques and technology. UNITED 2016 [https://unitedsummit.org/?CS=blog] is created from the ground up to provide the insight you need to drive your security program forward, faster. This year, we're also hoping you can provide us with the insight we need to make our products and services even better. That's why we're running two UX focus groups on November 1, 2016. We'd love to see you there—after all, your feedback is what keeps our solutions ever-evolvi

5 min IT Ops

Moving away from MVC

In of all my years as a software engineer, trying new libraries, frameworks and paradigms has been such a pleasure especially in web development. Even before the well known javascript libraries, web development was based on backend apps which render heavy html code within css and some js code. Frameworks such as spring, .NET MVC, django and rails helped us with abstractions and predone tasks increasing development speed and quality (reuse principles). But, it was not enough. Apps were getting

5 min IT Ops

Avoid These Things When Logging From Your Application

It seems almost strange to talk aboutavoiding things while logging. After all, logging is your last line of defense or your salvation in many cases. Some crazy bug in the field that shows up every third full moon? An external auditor looking at your app’s runtime behavior? Logging to the rescue. So naturally, is stands to reason that you would want to log just about *everything *your application. Whenever there’s any doubt, slam a logger call in there and let log level sort out the detai

12 min IT Ops

Designing Data Access Object (DAO) Layer

Overview / Motivation A key part in moving the creation of a product from custom craftsmanship to a repeatable engineering process is the construction of a modular system with decoupling of its components, as well as these components being easily adaptable to inevitable changes. In this blog, I will draw on some of my recent experience of architecting a DAO layer as part of a software system, and how I tried to ensure the development of this system as a healthy and sustainable environment that

5 min Log Search

Log Search Simplified

Hi, I'm Laura, UX Designer at Logentries and today I'm going to discuss how just about anyone can use Logentries to search and analyze their log data no matter what their job title or technical skill level. What is Logentries? At Logentries, the team works tirelessly to provide an easy to use log management service [https://logentries.com/?CS=R7blog] that allows users to stream their logs from just about anything. Logentries can accept data from almost any device that generates log data, inclu

5 min IT Ops

Get your work done even faster with the Logentries REST API

Now you can get your work done even faster by automating tasks with the Logentries REST API. With the ability to programmatically query data, manage users, create alerts and integrate third party tools, it’s now easier to finish the job and get on with your day. Table of contents * Query API- Example Usage * Team and User Management API- Example Usage * Tags and Alerts API- Example Usage * LeExportPy- Read More * LeCLI- Read More On-demand Webinar Interested in learning more about the

5 min IT Ops

Hashing Infrastructures

Engineers in fast moving, medium to large scale infrastructures in the cloud are often faced with the challenge of bringing up systems in a repeatable, fast and scalable way. There are currently tools which aid engineers in accomplishing this task e.g. Convection, Terraform, Saltstack, Chef, Ansible, Docker. Once the system is brought up there is a maintenance challenge of continually deploying and destroying the resources. What if we can hash the inputs for describing an infrastructure, where

5 min IT Ops

Logging the Lights in Your Home

It’s all the rage these days under the general heading of “Internet of Things” (IoT), but I have been a home automation enthusiast for more than 10 years now. In the interceding time, I’ve done experiments and written about the subject. I even published a Pluralsight course [https://www.pluralsight.com/courses/home-automation-fundamentals], in which I turned a Raspberry Pi into a RESTful server that lets you turn lights in your house on and off using basic X10 technology [https://en.wikipedi

3 min Log Management

Using Log Data as Forensic Evidence

This is a guest post by Ed Tittel. Ed, a regular contributor to blog.logentries.com [https://blog.logentries.com/], has been writing about information security topics since the mid-1990s. He contributed to the first five editions of the CISSP Study Guide (Sybex, 6e, 2012, ISBN: 978-1-119-31427-3) and to two editions of Computer Forensics JumpStart (Sybex, 2e, 2011, ISBN: 978-0-470-93166-0), and still writes and blogs regularly on security topics for websites including Tom's IT Pro, GoCertify.co

3 min IT Ops

4 Potential Security Issues Raised By Pokémon Go

Pokémon Go is a phenomenon. The game is objectively a success and has been breaking mobile gaming records almost weekly. The game’s current success is without being open in some significant markets and it shows no signs of slowing. It is important to remind players to take measures to protect your company’s interests when playing. Pokémon Go is an Augmented Reality game. Players see the game’s fictional world on top of everyday reality. Augmented Reality manifests in several ways: from import

3 min Log Management

Nexpose Logging Analytics using LogEntries

This blog shows how to use the power of LogEntries [https://logentries.com/] Search and Analytics to monitor your Nexpose installation. LogEntries has joined [/2015/10/13/why-we-re-welcoming-logentries-to-the-rapid7-family-a-story-of-data-and-analytics] the Rapid7 family and offers several powerful capabilities to search, analyze, monitor and alert on your Nexpose installation. LogEntries is also super easy to set up and maintain. I spent about five minutes getting it running. The Nexpose engi

5 min IT Ops

3 Ways to Improve Your Code Testing

I am a big fan of Test Driven Development [https://en.wikipedia.org/wiki/Test-driven_development] (TDD). I drank the kool-aid a while back and have not had a regret since. When I sling code, I am always writing a test, or writing against a test. That’s how I’m built. I am of the firm belief that one of the best ways to ensure short term and long term code quality is to make sure there that your tests are exercising as much of the code base as possible, as often as possible. Thus, I am always

1 min IT Ops

Integrating Logentries with OpsGenie: 3 Easy Steps

Real-time alerts [https://logentries.com/product/alerting-and-reporting/] are only as good as their ability to successfully reach their intended audience. If an alert recipient only checks email once every several hours, email alerts would not be well suited for real-time notification. It’s for this reason that Logentries makes it easy to integrate with popular 3rd party tools that DevOps professionals are already using, including Slack [https://docs.logentries.com/docs/partner-notifications#

4 min IT Ops

Exporting Logentries data with Leexportpy

Leexportpy, the Logentries utility that exports your log data to 3rd parties, has built-in support for various services such as Kafka, Geckoboard and Hosted Graphite. Without any modification to the current code, you can use these services to extract your Logentries data. To begin, make sure your read-write or read-only API key is correctly placed in the LE section of your configuration file as shown below. Also make sure you have read the first blog post of this series. [/2016/07/introductio

7 min IT Ops

What exactly is an Event-loop?

“The price of reliability is the pursuit of the utmost simplicity” – C.A.R Hoare Rather than doing another all-out performance post, I’ll look at some aspects of asynchronous I/O today instead: what it is at a high level, what it isn’t and why you would use it. There aren’t many aspects of programming today that are as saturated with buzzwords and misinformation as asynchronous IO and some of the frameworks which build on top of this. If you work with server code which has to handle a nontri

10 min IT Ops

Introduction to the Logentries Command Line Interface

The Logentries Command Line Interface (CLI) allows you to both manage and use your Logentries service right from the command line. The CLI is built on the Logentries REST APIs [https://docs.logentries.com/docs/rest-api] and provides a tool to interact directly with the Logentries service outside of the UI. It is in beta and currently supports retrieving log events, and performing queries and calculations on log events using our powerful querying language LEQL [https://docs.logentries.com/docs/

4 min IT Ops

Using Logentries With Angular v1.5

The post assumes at least a basic knowledge of Angular. Angular is a very opinionated framework so make sure you have some experience with Angular before following the instructions presented below. Logentries can integrate into whatever Javascript framework you want to use. Previously, we examined adding Logentries to a React application [/2016/05/integrating-the-logentries-javascript-library-with-react/]. This post will illustrate how to add Logentries to your Angular v1 application using

3 min IT Ops

REST API: a little cURL and some Python

Here at Logentries [https://logentries.com/centralize-log-data-automatically/?le_trial=rest_api_curl_and_python-logentries_blog-post_cta-create_trial&utm_campaign=rest_api_curl_and_python&utm_source=logentries_blog&utm_medium=post_cta&utm_content=create_trial] work has been going for sometime in bringing to our customers a powerful and flexible REST API [/2014/09/the-abcs-of-rest/] service for interaction with their log data. This work started out with the REST Query API [/2016/05/now-availab

3 min IT Ops

Backup Log Checks and What They Can Tell You

There is simply no substitute for a recent, accurate backup when it comes to recovering from file or system damage or outages. But that backup must be complete and error-free to make a full recovery possible. That’s why inspecting log files from backups is a critical and important step in verifying their accuracy or coverage, and a necessary check before performing a restore that converts any backup image or files into production status. Your backup logs [https://logentries.com/centralize-log

5 min IT Ops

Keep Your Code Clean while Logging

In my consultancy practice, one of the things that I do most frequently is help teams write so-called “clean code.” Usually, this orients around test-driven development (TDD) and writing code that is easily maintained via regression tests and risk-free refactoring. Teams want to understand how to do this, and how to do it in their production code (as opposed to in some kind of toy “let’s build a calculator” exercise). One of the most prominent, early sticking points that rears its head tend

6 min IT Ops

Rsyslog TCP/IP log forwarding to Logentries with Netscaler

This guest post originally appeared on IBM developerWorks [https://developer.ibm.com/recipes/tutorials/rsyslog-tcpip-log-forwarding-to-logentries-with-netscaler/] . Overview The aim of the article is to cover technical areas on how to load balance Rsyslog TCP/IP traffic from Redhat Server to Logentries (a third party solutions for real time log management, monitoring and analytics) and configuration used for load balancing network traffic flow between the connected servers. Majority of virtu

3 min IT Ops

Introducing LEQL: LIMIT

The Logentries query language, also known as LEQL, offers a very expressive way to query log data. If you’re unfamiliar with LEQL you can get a nice overview from one of our previous blog posts here [/2015/06/introducing-leql/]. Today we’re excited to announce the latest extension of LEQL’s capabilities – Limit. If you want to skip reading about this and just go try it out for yourself then head over to theadvanced view in LEQL Querybuilder [https://docs.logentries.com/docs/search#section-leq

5 min IT Ops

Introduction to Leexportpy

What is Leexportpy? Leexportpy [https://github.com/logentries/leexportpy] is a flexible and extensible Python application that enables log search results to be easily exported to third party services such as Geckoboard and HostedGraphite. The application utilizes the Logentries REST API [/2016/05/now-available-rest-query-api/] providing you with the ability to easily create queries on any of your log data, transform the results to the data format needed by the third party service and push tran

9 min IT Ops

Self-describing Logging Using Log4J

Over the last few months I’ve been writing about the benefit of using self-describing logging to make your log data easier and faster to process. I covered .NET [/2016/04/how-to-ensure-self-describing-log-data-using-log4net/] and NodeJS [/2016/05/implementing-self-describing-log-data-using-nodejs/]. In this article I am going to apply my thinking to Java and Log4J. I am going to show you a technique for creating self-describing logging in the JSON format. In order to get full benefit from r

4 min IT Ops

How Audit Logs Help Confirm and Correct Security Policy

There are many possible definitions for the term “security policy,” but all of them share certain elements in common. A security policy should lay out what assets, both physical and digital, an organization wishes to protect. It should explain what it means to be secure and to behave securely. In short, a security policy identifies what assets are to be protected, what kinds of risks such protection is meant to defeat or mitigate, and how security can be established, measured, and monitored. A

6 min IT Ops

Signal AND Noise The Best of All Worlds for Logging

One of the absolute, classic pieces of advice that you’ll hear when it comes to logging is what I think of as the iconic Goldilocks logging advice. It goes something like this. When it comes to logging, you don’t want to miss anything important because logging helps you understand your application’s behavior. But youalsodon’t want to log too much. If you log too much, the log becomes useless. You want to log just the right amount. Sage advice, to be sure. Right? Or, maybe, when you sto

20 min IT Ops

SSH from the ground up

If you work professionally in the IT industry, chances are you’ve been using OpenSSH for a long time now for your day to day work. OpenSSH however provides so much more than “just” remote shell on *nix system (and apparently on Windows [https://github.com/PowerShell/Win32-OpenSSH] too now!) and in this article we’re going to explore some of the non immediate uses of ssh and introduce a few accessory tools that make using ssh even better. Conventions for the examples We need to set some termi

4 min IT Ops

Migrating a web app to Angular

At some point many applications get to a state in which a large refactoring or in some cases a complete rewrite needs to happen. The decision to do so can be driven by many factors. For example, the code base is growing rapidly and the current architecture cannot support the growth, components are becoming too tightly coupled and need to be split, new and better technology becomes available which offers significant improvements or due to other factors the current code base is just not maintain

3 min IT Ops

Webinar Recap: Tableau Server Log Analytics

Our webinar [http://info.logentries.com/tableau-pluralsight-logentries-webinar-2016] was broadcasted & recorded on June 16th 2016. During this broadcast Tableau Zen Master Mike Roberts of Pluralsight discussed how to develop a simple technology stack for next-gen management of Tableau using Logentries [https://logentries.com/centralize-log-data-automatically/?le_trial=tableau_webinar_june16_recap-logentries_blog-post_cta-create_trial&utm_campaign=tableau_webinar_june16_recap&utm_source=logentr

11 min IT Ops

Monitoring SNS Activity Using a Lambda Function and Logentries

Amazon Web Services Lambda functions are very cool. A Lambda function is a feature in Amazon Web Services that allows you to put a discrete piece of computing logic up in The Cloud and then access that logic to meet a particular need. For example, you can create a Lambda function that takes a list of stock symbols and does some analysis on the list using other cloud based services in order to suggest the best stock to buy. Logentries has put the power of Lambda functions [https://docs.logentrie

3 min IT Ops

Checking Active Directory (AD) Security and Integrity via Log Monitoring

Because AD literally holds the keys to the kingdom for domain-based networks and runtime environments, it’s usually a good idea to keep a close eye on those keys, how they’re being used and what kinds of attacks might be directed at them. Monitoring AD related event logs provides one great tool in exercising due security and integrity diligence. Alerting on specific high-impact events provides yet another. For over two decades now, Microsoft’s Active Directory (AD) has provided a powerful set

3 min IT Ops

Exploiting Zookeeper for managing processes in a production environment with Lockex

Lock and execute! As an engineer here at Logentries [https://logentries.com/centralize-log-data-automatically/?le_trial=exploiting_zookeeper-logentries_blog-post_cta-create_trial&utm_campaign=exploiting_zookeeper&utm_source=logentries_blog&utm_medium=post_cta&utm_content=create_trial] I need to maintain a complex system that has requirements for being available to our customers. We always build systems with the ability to be resistant to failure. In our environment, we have processes and dae

2 min IT Ops

Introducing LEQL: BYTES

Take a Byte out of Anomalous Log Volumes One of the most common questions asked by users of our Log Management platform [https://logentries.com/centralize-log-data-automatically/?le_trial=leql_bytes-logentries_blog-post_cta-create_trial&utm_campaign=leql_bytes&utm_source=logentries_blog&utm_medium=post_cta&utm_content=create_trial] is “where is my log volume coming from?” There are a number of ways to interpret this question. Do I have a log source that is sending more events than usual? D

8 min IT Ops

A different way to log your website usage

Our recently published whitepaper [http://info.logentries.com/15-events-to-log-for-better-devops?le_tofu=15-events-to-log-for-better-devops-blog-popup-view_lander&utm_campaign=15-events-to-log-for-better-devops&utm_source=blog&utm_medium=popup&utm_content=view_lander] demonstrates some of the more business-orientated metrics that you can log so you can get a better insight into your interactions with customers. There may also be some Sales & Marketing type interactions with potential customer

1 min IT Ops

Tableau Server Log Analytics: Easily Monitor Your Data Analytics Stack

Over a few short years, Tableau has changed the way employees use data to make better business decisions. Thanks to Tableau’s robust set of admin capabilities, administrators are now empowered to change the way traditional business intelligence software is managed. Join Tableau, Pluralsight and Logentries on Thursday, June 16, 2016 at 2:00 PM EDT as we explore how to develop a simple technology stack for next-gen management of Tableau. Webinar Details Date: Thursday, June 16th, 2016 @ 2:00 P

5 min IT Ops

Logging for Continuous Integration

If you look at the title of this post, you’re probably thinking to yourself, “huh, that’s never really come up.” Of course, it’s possible that you’re not. But, in my travels as a consultant helping dev teams with practice and gap analysis, I’ve never had anyone ask me, “what do you recommend in terms of a logging solution for continuous integration?” But hey, this is an easily solved problem, right? After all, continuous integration meansJenkins [https://jenkins.io/], andJenkins has an appl

4 min IT Ops

Webinar Recap: How to Easily Send and Use your IoT Data in the Cloud

Our webinar [http://info.logentries.com/how-to-easily-send-and-use-your-iot-data-in-the-cloud-webinar] was broadcasted & recorded on May 25th 2016. During this broadcast, Rapid7 Consulting Software Architect David Tracey discussed how to easily capture, aggregate, analyze and alert on your IoT data using Logentries. Below is a brief summary of some of the topics David covered. To watch a recording of the full webinar on-demand, click here [http://info.logentries.com/how-to-easily-send-and-us

1 min IT Ops

Integrating Logentries With .NET The How and Why

A robust logging strategy opens up a world of potential improvements for your .Net applications through application logging. Application logging provides valuable insight. Insight that can only benefit your network application stack since your .Net application is the front line for enhancing your customer’s experience. Bringing meaning to all the potential information that your .Net application can collect is what Logentries does best. Logentries makes getting this valuable information into y

4 min IT Ops

Intrinsically fast: more JVM performance tinkering

I didn’t expect my last post [/2016/03/14/a-point-of-contention-cache-coherence-on-the-jvm/] on JVM perf to be so well received, so I thought I’d carry on digging into why your code does (or doesn’t) run fast! Let’s forget about concurrency for now and instead focus on the executable machine code that the Java Virtual Machine (and particularly HotSpot) generates. In Java-land it’s pretty common to hear people mention stuff about ‘warmup times’, especially in the context of an incendiary micro

2 min IT Ops

Now Available: REST Query API

[/2016/05/now-available-rest-query-api/] Our new REST Query API is now included as part of the Logentries Team plan [https://logentries.com/pricing/]. Complimentary access to the REST Query API for all active Starter and Pro accounts is also available for a limited time. Don’t have an account yet? You can try out all the features of Logentries (including the Query API) with our free 30-day trial [https://logentries.com/centralize-log-data-automatically/?le_trial=rest_query_api-logentries_blog-

7 min IT Ops

Implementing Self-Describing Log Data Using NodeJS

In my previous article, How to Ensure Self-Describing Log Data using Log4Net [/2016/04/how-to-ensure-self-describing-log-data-using-log4net/], I showed you a technique that made structuring your logging information as key-value and JSON under Log4Net an easier undertaking. In this article I am going to apply the same concepts to NodeJS. I’ll show you how to make it so that so you can ensure logging uniformity among all the NodeJS developers in your enterprise with little to no additional effo

3 min IT Ops

How To: Send Logentries Alerts to BigPanda

Working in customer support we are usually the first to receive feature requests, integration requests or recommendations. We would then relay this to our product team. But we often get requests that we can tackle ourselves whether this may be small coding tasks or account changes. So when we heard a user wanted to be able to forward their Logentries alerts to BigPanda.io, we made that happen. When any issues occur, time to resolution matters. Because Logentries streams your log data in real-

5 min IT Ops

Raspberry Pi, Logs and IoT - Sending Pi Log and Sensor data to Logentries

In the previous blog post [/2016/02/iot-made-real-using-ti-sensortag-data-with-logentries/] we learned how to send IoT data from the TI CC 2650 SensorTag to Logentries using Node-Red and directly using Linux. This Blog will show how to send data from a Raspberry Pi device to Logentries [https://logentries.com/centralize-log-data-automatically/?le_trial=raspberry_pi-logentries_blog-post_cta-create_trial&utm_campaign=raspberry_pi&utm_source=logentries_blog&utm_medium=post_cta&utm_content=create_

1 min IT Ops

Webinar: How to Easily Send and Use your IoT Data in the Cloud

Rapid7 Consulting Software Architect David Tracey spends his days working on the Logentries product creating a Highly Scalable Log Storage, Analytics and Querying System with emphasis on introducing RESTful Architecture and APIs. David has published a series of blog posts about working with IoT [/category/iot/] data in the cloud. Join [http://info.logentries.com/how-to-easily-send-and-use-your-iot-data-in-the-cloud-webinar] David Tracey and the Logentries by Rapid7 team on Wednesday, May 25,

4 min IT Ops

A Query Language for Your Logs

Application logging is the software world’s version of archeology. At runtime, your application lives in a rich, colorful, 3-dimensional world of flowing aqueducts, packed coliseums, and bustling streets. There’s more going on than can possibly be captured. When you’re trying to reproduce and correct a reported issue, you play archeologist. The vibrant, live world is gone, and you’re left to piece reality back together using only decorated pots, spearheads, and fragments of frescoes. In oth

6 min IT Ops

Integrating the Logentries Javascript Library With React

React.js has proven itself a powerful contender in the world of Javascript frameworks. Arguably, it has become one of a handful of libraries that all web developers should consider for current or upcoming projects. Understanding how it integrates with other libraries in your technology stack is an important part of that consideration. If you currently use, or are considering using Logentries [https://logentries.com/centralize-log-data-automatically/?le_trial=react-logentries_blog-post_cta-crea

5 min IT Ops

Logs for SEO

I write blog posts on a number of different sites that are not my own, and that is an exercise in pure writing joy. I compose the posts, I submit them, and voila! They’re published on nice-looking sites, promoted by people with reach, and read by many (hopefully) interested readers. Life is good. By way of comparison, when it comes to my own blog, life is not quite so simple. On my own blog, I have to write the posts and manage all of the details that are abstracted away when I write for o

6 min IT Ops

Queuing tasks with Redis

Overview As stated on their official homepage [http://redis.io/], Redis is an open source (BSD licensed), in-memory data structure store, used as database, cache and message broker. Little bit about what Redis can do. It supports data structures such as strings [http://redis.io/topics/data-types-intro#strings], hashes [http://redis.io/topics/data-types-intro#hashes], lists [http://redis.io/topics/data-types-intro#lists], sets [http://redis.io/topics/data-types-intro#sets], sorted sets [http:/

5 min IT Ops

The Power of IOT with a Query Language

Using Logentries Query Language (LEQL) to analyze Sensor Data This is one of a series of IoT blogs that show how easy it can be to integrate a range of real sensor devices with Logentries. The focus of this blog is to show how the Logentries Query Language (LEQL) can be used to search and analyze that data. A separate blog will also show the usefulness of the alerting functionality in Logentries, where labels can be set that will display with matching events/data, alerts can be sent to a range

12 min IT Ops

How to Ensure Self Describing Log Data Using Log4Net

In a previous article, The Benefit of Having an Enterprise Logging Policy [/2016/04/06/the-benefit-of-having-an-enterprise-logging-policy/], I presented the case for always using self-describing data formats when logging information. Using self-describing formats, such as key-value pairs and JSON, saves time and effort in terms of indexing and subsequently querying your logs on the backend. Also, logs that use a self-describing data format are easier to understand by anyone, at any time. In t

2 min IT Ops

15 Events to Log for Better DevOps

DevOps practices are now finding their way into more and more organizations. Up to now, as the DevOps movement has started to percolate across various industries the focus has been on improving the IT department. In InsightOps’ article15 Events to Log for Better DevOps [https://information.rapid7.com/15-events-for-better-devops.html]we discuss how the effective application of DevOps principals can benefit every department in the organization. Below are some excerpts from the full article, avai

8 min IT Ops

Using JavaScript to interact with the REST Query API

We’re very excited to announce that our REST Query API is now available [/2016/05/now-available-rest-query-api/]. With this API, you can: * make it easy to remotely query your log data * easily integrate Logentries with third party solutions, external systems and internal tools * allow users and systems to query their log data programmatically over our REST API In this article, I will show how you can quickly interact with the Query API by sending in a LEQL query [https://logentries

3 min IT Ops

How to: Send SMS messages to Logentries in under 5 minutes (maybe 10)

The “Internet Of Things” continues to be talked about a lot with an increasing number of devices now containing some sort of smart functionality which can be interacted with. Here’s a great article about end-to-end IoT monitoring [/2014/12/end-to-end-iot-monitoring-with-log-data/] by colleague David Tracey. However, not all IoT devices can be in locations with WiFi or 3/4G coverage, so they can not easily (or at all) send or receive data over the internet, and instead rely on standard cellula

4 min IT Ops

How to Log from Azure Virtual Machines

You have evaluated the many IaaS providers [http://info.logentries.com/how-to-compare-google-compute-engine-and-aws-ec2?le_tofu=LogFromAzure-leblog] out there and you have decided on Azure Compute [https://azure.microsoft.com/en-us/documentation/scenarios/virtual-machines/]. Great choice! Azure is an ideal provider with broad support for various operating systems, programming languages, frameworks, tools, databases and devices. Azure also has the unique ability to facilitate hybrid deploymen

15 min IT Ops

How to Compare Google Compute Engine & AWS EC2

Which Virtual Machine is Best: Google’s Compute Engine or Amazon’s EC2? It Depends. The Internet might seem like a Wild West of chaotic connections because it often is. Companies like Google and Amazon have been managing to create order out of the chaos for years by understanding the nature of the World Wide Web. Within the last 10 years, Google and Amazon have leveraged that understanding into a robust suite of product offerings in the field of Infrastructure-as-a- Service, or IaaS. The corn

3 min Metasploit

Securing Your Metasploit Logs

Metasploit, backed by a community of 200,000 users and contributors is the most impactful penetration testing solution on the planet. With it, uncover weaknesses in your defenses, focus on the highest risks, and improve your security outcomes. Your Metasploit Pro console produces a lot of important logs. It is essential to be able to review these logs, alert on them, and keep them secure. Why should I monitor these logs? The logs produced by your Metasploit Pro console are helpful when troubl

7 min IT Ops

The Benefit of Having an Enterprise Logging Policy

Lack of an enterprise logging policy is a common shortcoming when it comes to the organizational discipline of logging from within large, distributed applications. Just because you can get log data into a system, it does not necessarily follow that the data you are entering is useful. The old adage, garbage in, garbage out holds true. If an enterprise allows anybody to enter log data in any way possible, anybody will. In the long run, without a proper policy for logging, and procedures to supp

5 min IT Ops

Client Side Logging In Javascript

Developers are writing Javascript applications of increasing complexity designed to run in web browsers, on desktops, and on servers. Javascript applications have reached a level of maturity that means they are running important business operations. They must be more maintainable and supportable now that they have achieved this level of responsibility in the enterprise. Javascript applications should be expected to provide the same information for support and maintenance as any other applic

1 min IT Ops

Infographic: What scares IT Professionals most about IT Infrastructure

Download Now [http://info.logentries.com/logentries-download-infographic-it-infrastructure] Logentries surveyed IT Professionals identifying themselves as members of Information Technology Teams, Operations Teams, and Development Teams Asking them: “What scares you the most about your IT infrastructure?” The answers highlighted Security concerns, System Failure, Operational costs, and the complexities of SDN (Software Defined Networking). We saw many responses reinforcing the need to conti

6 min IT Ops

Do You Still Email Yourself from Your Code? How to Stop the Madness

A few years back now, I took on an assignment to help a company modernize a series of legacy .NET applications. One of these did some back office processing. A vendor would stick some files on a shared drive, and a windows scheduled task would invoke this bit of code to parse the file, apply a whole slew of business rules to its contents, and then update the appropriate internal systems. The details are both proprietary and uninteresting, so I will spare you those. The author of this appli

3 min IT Ops

Webinar Recap: Logentries Feature Walkthrough

Our March webinar was broadcasted & recorded on March 16th 2016. This broadcast included a Logentries demonstration by Mike Neville-O’Neill, followed by a live Q&A lead by Justin Buchanan and Matt Kiernan. The webinar recording is available here [http://info.logentries.com/logentries-feature-walkthrough-march]. Below is a summary of some of the features covered. To follow along, or to try the features for yourself, be sure to create a free Logentries account [https://logentries.com/get-start

13 min IT Ops

The 4 Steps for Creating a Log Enabled Marketing Campaign

Typically, most logging activity in the online world is concerned with collecting information about an enterprise’s digital infrastructure. Machine logs, application logs, network logs, database logs, access logs are a few examples of such activity. However, as marketing campaigns become more integrated into application activity, using log data to monitor and to measure the effectiveness of a campaign is a viable extension of an enterprise’s current logging activity. But, we need to beware.

5 min IT Ops

Brics Vs RE2/J

By Benoit Gaudin and Mark Lacomber Regular Expressions When it comes to searching unstructured data, regular expressions [https://logentries.com/doc/regex/] are a very useful and powerful tool. The power provided by popular regular expression libraries does come with a significant performance cost in some cases though, both when compiling regular expressions into automata (state explosion problem when determinising automata, as illustrated on some examples here [http://www.microarch.org/micro4

5 min IT Ops

A point of @Contention- cache coherence on the JVM

Java 8’s major changes- lexical closures, the stream API, e.t.c have overshadowed a slew of little gems, one of which I only discovered the other day- the @Contended annotation. False Sharing Chances are you’re reading this on a device with more than one CPU. There’s therefore also quite a good chance the you have more than one thread of execution running at the exact same time. There’s an equally good chance that some of your fancy multiprocessor CPU’s on-die memory (aka L2/3 cache) is share

4 min IT Ops

Deciphering MySQL Logs: The What, Why, and How

Logs are one of the best ways to understand what a server is doing. Thankfully, MySQL has no shortage of log activity to assist a DBA in its maintenance. It writes out its activity to 5 different logs. This post will take a look at the existing MySQL logs and how they assist the administrator. * On Windows, - The log is written to the data directory with a .err extension even if not explicitly enabled. * Errors are automatically written to the Event Log. This behavior is standard and

3 min IT Ops

How to Analyze Heroku's New Runtime Metrics

Heroku Labs [https://devcenter.heroku.com/categories/labs], Heroku’s platform for providing “experimental features that are under consideration for inclusion into the Heroku platform”, recently launched <a href="https://devcenter.heroku.com/articles/log-runtime-metrics" target="_blank">log-runtime-metrics</a> for “enabling visibility into load and memory usage for running dynos”. With log-runtime metrics, Heroku now inserts the following per-dyno stats directly into the log stream: * Memory

8 min IT Ops

7 Rules for Using Log Data Effectively in a Retrospective

Log data [https://logentries.com/insights/log-data-centralization/] can be an indispensable tool for doing an effective Retrospective following a technical disaster. Yet, often the data is misused. And many think that the entire Retrospective process is flawed altogether. More often than not Retrospectives, also known as Post Mortems, turn into technical autopsies. A bunch of people get together to find out how the system died and who is responsible. Data becomes nothing more than evidence of

3 min IT Ops

New Automated Log Parsing

The Logentries product is always improving and advancing. There are some exciting new features available today, and another great feature coming this month! Included in today’s announcement: 1. Automated Log Parsing * Apache and Nginx logs * Syslog Tags 2. Nested JSON Support -------------------------------------------------------------------------------- Automated Log Parsing If you’ve ever sent data to Logentries you may have noticed keys being automatically highlighted and availa

10 min IT Ops

Logging for Unikernels

Unikernels are the next step in virtualized computing. There is a lot of hubbub right now in the tech-o-sphere about unikernels. In fact, in many circles unikernels are thought to be the Next Big Thing. There is a lot of recent news to justify this perspective. Docker, a key player in Linux Containers, has acquired Unikernel Systems. MirageOS, the library that you use to create unikernels has solid backing from Xen and the Linux Foundation as an incubator project. Unikernels are not going any

2 min IT Ops

Logging from Sonos Speakers with a Webhook

Our Boston sales office has a big set of Sonos [https://www.sonos.com/] speakers around their office that pumps tunes out across the office. It’s great for camaraderie, until someone sticks country music on… On a visit, I had an hour to spare, so I decided to find out if I might be able to log the music activity to Logentries, and it was surprisingly simple. After some quick googling, I found a few node.js libraries that interface with Sonos, and the Sonos HTTP API [https://github.com/jishi/n

6 min IT Ops

The yellow brick road to machine learning with honeypot data: Our lessons learned

Recently the Rapid7 Logentries [https://logentries.com/get-started/] team attended a hackathon over at one of our Boston offices. This was a great way for us to integrate with the other Rapid7 teams within the company and to have fun messing around with things we don’t usually have time for in a working day. The project that my team worked on involved machine learning with the dataset collected by some of the various Heisenberg honeypots that Rapid7 has deployed. More information about these

3 min IT Ops

Webinar Recap: Logentries Feature Walkthrough

Our second February webinar was broadcasted & recorded on February 25th 2016. During this broadcast, Justin Buchanan demonstrated many of the essential Logentries features. Below is a summary of some of the features covered. To follow along, or to try the features for yourself, be sure to create a free Logentries account [https://logentries.com/get-started/?le_trial=blog-febwebinarwalkrecap]. Query Log Data Once you successfully get data [https://logentries.com/resources/how-to-videos/sendin

5 min IT Ops

The Role of Log Files in Experiments

You have heard, no doubt, of theLean Startup [http://www.amazon.com/The-Lean-Startup-Entrepreneurs-Continuous/dp/0307887898/] . If you need a refresher to place the name, it’s a book, but it’s also a business trend with such momentum as to have awebsite advertising it as a “movement.” [http://theleanstartup.com/]And, frankly, that advertisement is hardly a stretch. The title and the terms coined in it are on everyone’s lips in the tech industry these days because people at companies of all s

2 min IT Ops

Infographic: Log Management & Analytics at a Glance

Logentries surveyed over 400 IT professionals about their views of log management [https://logentries.com/get-started/?le_trial=survey-infographic-blog] and analytics. We found that many people are planning on using and investing in public cloud infrastructure such as AWS, Google [/2016/02/how-to-compare-google-compute-engine-aws-ec2/], and Azure in 2016. Who took the survey? 40% of respondents identify as a Developer / Dev Manger, as we expected. We found the 3% of responses from Support and

5 min IT Ops

TypeScript Language Primer

What is TypeScript? TypeScript is an open source typed superset of JavaScript [https://logentries.com/doc/javascript/] which compiles to plain JavaScript. Many people have considered JavaScript’s loose typing as a problem, but now TypeScript offers a solution to that. Specifically, TypeScript allows you to code with decorators/annotations, classes, interfaces, private properties and typing compliance. We also might say that TypeScript is ES6 with some extra options. What does TypeScript do?

8 min IT Ops

IOT made real - Using TI SensorTag data with Logentries and NodeRed

Learn how to send IoT Data from the TI CC 2650 SensorTag to Logentries (using Node-Red). This is the first of a series of IoT blogs that show you how easy it can be to integrate a range of real sensor devices with Logentries and how to use the data from those devices once it is in Logentries. This follows the earlier blog [/2014/12/end-to-end-iot-monitoring-with-log-data/] showing why a centralised logging service would be useful for IoT developers and users. This series of blogs will show ju

5 min IT Ops

Troubleshooting with Nexpose Logs

Nexpose [http://www.rapid7.com/products/nexpose/index.jsp] is the industry standard in Vulnerability Management, giving you the confidence you need to understand your ever-changing attack surface, focus on what matters, and create better security outcomes. [/2016/02/troubleshooting-with-nexpose-logs/] Table of contents * Where are the Nexpose logs located? * Setting up the Logentries Agent * Analyzing the logs- mem.log * nsc.log * auth.log * Get started -------------------------------

3 min IT Ops

February Webinar Recap: Monitoring Business Critical Events

Our firstFebruary webinar [http://info.logentries.com/monitoring-business-critical-events] was broadcasted & recorded on February 11th 2016, in conjunction with VictorOps [https://victorops.com/blog/logentries-webinar/]. During this broadcast Jason Hand (VictorOps), Matt Kiernan (Logentries), and Justin Buchanan (Logentries) discussed effectively monitoring for business critical events and metrics. Below is a brief summary of some of the critical events reviewed. To watch a recording of the f

3 min IT Ops

Using Logs for Security & Compliance: Part 3

This 3-part series explores the critical role logs play in maintaining regulatory compliances and provides specific examples of known events to look for an how to evaluate different compliance tools. To download the free 24-page white paper, click here [http://info.logentries.com/using-logs-to-address-compliance-standards]. -------------------------------------------------------------------------------- [http://info.logentries.com/using-logs-to-address-compliance-standards] When it comes to

5 min IT Ops

Considering the Explosive Growth of Log Analytics

You’d have to be living in a cave to not know that the practice of log analytics in corporate IT has grown dramatically in the last 10 years. This explosion in logging activities over the recent years is due to two factors, the maturing of log technology and the expanded application of logging to new information domains such as tracking user behavior, tracking page views, and tracking API interaction, to name a few such activities. As logging technology matures, the price goes down. Getting a

1 min IT Ops

Video: Logs for PCI Compliance

Watch Video [https://www.rapid7.com/resources/using-logs-for-pci-compliance/] PCI DSS provides some specific requirements for how logs should be collected, reviewed and retained to maintain compliance. In Rapid7’s latest “Whiteboard Wednesday” video, I review three areas of PCI requirements for log management, including: * Why logs need to be stored in a secure location * The importance of regular investigation of known events * The necessity of audit trails provided by your log managemen

2 min IT Ops

Using Logs for Security & Compliance: Part 2

4 min IT Ops

Combining AngularJS and ReactJS for better applications

If you’re a JavaScript developer, and know a little bit about the current state of JS development, then you’re probably well aware that there seems to be a new JS framework popping up every day. It’s easy to feel overwhelmed with so many choices and that can make it difficult to actually choose the right tool for the job. This raises a question though, why do we have to settle on one tool? In recent years there’s been one topic of debate that’s drawn a lot of attention: React vs Angular. Whic

4 min IT Ops

January Webinar Recap: 2016 Best Logging Practices

In our January webinar [http://info.logentries.com/new-logging-practices-for-2016?utm_source=logentries&utm_medium=blog&utm_campaign=january_webinar] , broadcasted & recorded on January 28th 2016, we shared 5 best practices for log management & analytics in 2016 that were inspired by the most common questions we received in 2015. Below is a brief summary of those 5 best practices (plus one bonus practice). To watch a recording of the full webinar on-demand, click here [http://info.logentries

2 min IT Ops

Using Logs for Security & Compliance: Part 1

This 3-part series explores the critical role logs play in maintaining regulatory compliances and provides specific examples of known events to look for an how to evaluate different compliance tools. To download the free 24-page white paper, click here [http://info.logentries.com/using-logs-to-address-compliance-standards]. -------------------------------------------------------------------------------- For organizations that need to remain compliant with specific regulatory standards, requ

2 min IT Ops

Analyzing ELB Log Data

Thanks to some slick work from our engineering team, we have recently released a lightweight python script that will allow you to pull your Elastic Load Balancer logs from S3 into Logentries. In this implementation, we use AWS Lambda and leverage the S3 trigger, so the script only runs when needed. The full documentation is available here: https://logentries.com/doc/s3-ingestion-with-lambda/ [https://logentries.com/doc/s3-ingestion-with-lambda/?utm_source=134&utm_medium=blog&utm_campaign=12

1 min IT Ops

Introducing a Buildbot status plugin for pushing status updates to Logentries

Buildbot is a framework for building continuous deployment and integration systems, it is highly flexible and is written in python. It is also a mature system which a number of large projects use e.g. Mozilla, Chromium, Python – see trac.buildbot.net/wiki/SuccessStories [http://trac.buildbot.net/wiki/SuccessStories] To send build status information — specifically Start, Success and Failure states from Buildbot to Logentries — start by generating a log token from Logentries. [https://logentri

2 min IT Ops

How to Log Messages from Slack

We recently added support for unedited HTTP logging in Logentries. This means you can send us log data via HTTPS drain (from heroku), or via any webhook you want. One webhook that we’ve been looking to log for a while is Slack [https://logentries.com/resources/#plug-ins]. People are always chatting away on Slack, and this data might be useful some day. You can send the data into Logentries however you want, and then worry about what to do it when you actually need it! First, you’ll need to

3 min IT Ops

Logentries recognized by Docker as Ecosystem Technology Partner for Logging

Since last year, we’ve anticipated the impact of Docker [/2014/05/musings-on-the-future-of-docker/] and have been building integrations – first as experiments [/2014/03/how-to-run-rsyslog-in-a-docker-container-for-logging/] and later as full-blown solutions [/2015/07/an-all-inclusive-log-monitoring-container-for-docker/]. It’s therefore with great pleasure that we’re announcing our recognition by Docker as an Ecosystem Technology Partner for Logging. Why Monitor Docker Logs? Most teams that

5 min IT Ops

Analysing Hystrix metrics with Logentries

We’ve been using Hystrix [http://techblog.netflix.com/2012/11/hystrix.html] in production here at Logentries for over a year now [shameless plug: I briefly talked about this [https://speakerdeck.com/m0wfo/clojure-ireland-talk-june-2015] at a Clojure Ireland meetup recently :)] and have found it useful not only for bulkheading [http://martinfowler.com/bliki/CircuitBreaker.html] requests, but for getting fine-grained metrics for internal API calls. Netflix has also open-sourced a funky dashbo

3 min IT Ops

Introducing the Logentries callback plugin for Ansible

Ansible is an automation tool for deploying, configuring and orchestrating systems. It’s written in python and it is easily extended. A callback plugin for Ansible extends the functionality by being able to respond to events. These responses can be anything from executing tasks to logging information to different destinations. The Logentries [https://logentries.com/centralize-log-data-automatically/?le_trial=callback_plugin_for_ansible-logentries_blog-post_cta-create_trial&utm_campaign=callb

4 min IT Ops

Introducing LEQL: percentile() & median

While analyzing data, it’s important to use a variety of calculations to ensure you get the best insights. Today, we’re excited to announce the availability of our two newest LEQL functions: percentile() and median. percentile() allows you to calculate the number below which a given percentage of your log entries fall. To use a real world example, what was the longest response time for 95% of my application’s users? Similarly, median (or the 50th Percentile) gives you the middle number in a s

2 min IT Ops

October Feature Recap

As we practice continuous delivery at Logentries, we’re constantly deploying product updates of all sizes. Here’s a recap of feature updates from October! timeslice() timeslice() is our latest addition to the Logentries Query Language (LEQL) and gives you the ability to specify the granularity of your query results. By default, LEQL [/2015/06/introducing-leql/] queries that distribute data over a timeseries will do so over 10 data points. With timeslice(), you can specify how many data points

4 min IT Ops

Unleash the power of node.js for Shell Scripting (Part 2)

Ready for our first proper node.js Script! In a previous post [/2015/10/20/unleash-the-power-of-node-js-for-shell-scripting-part-1/], we learned about some tools that helped us create a script in node.js. It is now time to put this into practice by implementing a script that connects to a few online newspapers, searches in the news for specific keywords and returns those articles. Our new script will need to accept the following parameters: * A file with the list of newspapers (one URL per li

8 min IT Ops

Java 8: A quick introduction to Parallelism and the Spliterator

Introduction With the release of Java 8 a number of new language features were introduced [1] . These included lambda functions, streams and completable futures. Colleagues of mine have already reviewed these features in previous articles on this blog, which I recommend reading as part of this topic [2] [3]. In this article I will touch on an aspect of the Java 8 release that relates to the push towards exploiting parallelism, in the context of the existing Collections Framework – specifically

5 min IT Ops

Unleash the power of node.js for Shell Scripting (Part 1)

Setting Up Running a Node Script We are going to talk about creating shell scripts with node.js. The first thing that you need to do is install node.js. You can get the installers from https://nodejs.org/en/download/, or use your favorite package manager. Let’s go straight in and write our first script. The first thing to do is to create a file called script.js with the following code: console.log('hello world') We can now execute this script by running the following command in our shell:

3 min IT Ops

Logging from Tableau for Successful DataOps

Lately, we’ve been seeing a growing number of customers using Logentries in support of DataOps – the practice of collecting, normalizing and redistributing data throughout an organization so teams can make smarter business decisions. With teams ranging from engineers to support to marketing & sales relying on data for every day decision making, a critical requirement of those within a DataOps role is to actively monitor their organization’s data platforms. I recently had the pleasure of chatt

4 min IT Ops

Exploring Lambdas and Streams in Java 8

Java 8 introduced a host of new features, including lambda functions and streams. In this article I will focus on these two features as they are the most impactful features that were added in this new version of Java. Lambda Functions If you ever had to write a GUI for an application with an OO language you’ll understand the pain of writing event handlers, such as a mouse click event. So much boilerplate needed because you had to define a disgusting inline class with a single method. With the

6 min Log Management

10 Best Practices for Log Management and Analytics

Introduction Today’s Log Management and Analytics Challenges Within the last decade, the advancement of distributed systems has introduced new complexities in managing log data. Today’s systems can include thousands of server instances or micro- service containers, each generating its own log data. With the rapid emergence and dominance of cloud-based systems, we have witnessed explosive growth in machine-generated log data. As a result, log management has become a staple in modern IT operatio

1 min Logentries

Logentries Joins the Rapid7 Family

I'm very excited today to join the Rapid7 family. The acquisition is good news for Logentries customers, Rapid7 customers and all of our employees. It means that great minds and innovative technology have come together to solve some of our thorniest IT and security challenges. The Logentries team has been on a mission over the last few years -- Revealing the Power of Log Data to the World. While pursuing our mission, I am often asked why log data has become so valuable. The answer is simple: l

2 min Logentries

Why we're welcoming Logentries to the Rapid7 family - a story of data and analytics

Those that follow Rapid7 will know that we talk a great deal about our vision of delivering security data and analytics to our customers to enable an active, analytics-driven approach to cyber security. I'm excited to let you know that today we're making an important addition to the Rapid7 family that will help us advance this vision even further… we are acquiring the world-class, cloud-based log management and search technology company, Logentries. Organizations need real mastery of the inform

3 min IT Ops

Revealing Hidden Insights with Docker & timeslice()

Over the last few months, you’ve probably noticed that we’ve been talking about Docker quite a lot – we think this lightweight and powerful way of managing environments is only going to increase in popularity. Docker has been evolving their logging capabilities with each release, so of course we’re interested in how we can help you get log data from Docker into Logentries for quick and powerful analysis. In August, we announced a new way to collect and analyze Docker logs for free with our Doc

1 min IT Ops

Introducing LEQL: SORT()

If you’re familiar with a query language like SQL, you’re likely used to being able to sort your query results. When querying log data, sorting your results can come in handy when you want to analyze things like which Docker containers are using the most memory [/2015/07/an-all-inclusive-log-monitoring-container-for-docker/], or which URLs are being request most frequently from your CDN. Today, we’re announcing SORT as the latest function introduced into Logentries’ query language, LEQL. Much

7 min IT Ops

Do you need an Architect in a Software Company?

This may be a dangerous question to ask for someone whose role is that of an Architect, but I think it is a valid question for an Architect to ask. This is particularly true in the software industry where the role is interpreted in many different ways. In some cases, an Architect may work in an established enterprise company and hand down instructions on technology stacks to the developers. At the other extreme an Agile development team may work without the involvement of an Architect. Neither

7 min IT Ops

Log Analysis for Containers

Introduction The IT and DevOps world has come a long way with infrastructure. Virtualization revolutionized our ability to quickly deploy an application and scale up services when needed, paying only for the computing power used. Over the last few years, agile methodologies and continuous delivery have pushed VMs to their limits. Many teams still repeatedly use a single VM for releases and testing. Production VMs rarely change unless something goes seriously wrong. At the pace software develop

4 min IT Ops

Common Angular Routing Challenges

When it comes to frameworks, no one is perfect. As we migrate the Logentries application from legacy code to Angular, we’ve encountered a few interesting challenges along the way that we’ve enjoyed investigating and resolving. While specific challenges often depend on your project and migration strategy, the aim of this post is to share our solutions to problems one may encounter when migrating an app to Angular. In particular, I’ll focus on how Angular handles routing and some issues we’ve en

2 min IT Ops

Q&A with Sysdig's Chris Crane + Upcoming Webinar

This Thursday, September 17th, we’ll be teaming up with Sysdig to explore “container monitoring as a microservice” [http://info.logentries.com/container-monitoring-as-a-microservice] – the practice of achieving deep, container-level insight while following microservice best practices. During the webinar, Logentries’ Co-Founder Trevor Parsons and Sysdig’s COO Chris Crane will be providing live demos of what metrics you should be collecting and how you should be tracking these metrics. I had a ch

3 min IT Ops

Tips for Optimizing your AngularJS app

AngularJS can have performance problems when you start to scale your application. You may notice that your AngularJS application works well at first but as it grows in complexity, so does its load time. Maybe it hangs for a second (or more) on initial load and when its fully loaded, it lags. If this sounds familiar, this article is for you. There are several reasons your AngularJS application may be slowing down – here’s a list of solutions to the most common issues. Reduce Watchers Reduce t

1 min IT Ops

Best Practices for Container Log Analysis: Part 2

This 3-part series explores the challenges presented by containers and the advantages of using an end-to-end container log monitoring solution for complete container environment visibility. [http://info.logentries.com/log-analysis-containers] When working with containers, setting up a local image repository like Docker’s Registry can enable a team to iterate quickly, easily storing image versions in a central location to be used as needed. Yet as multiple team members update images, start, st

4 min IT Ops

Getting started with Docker, Google Container Engine & Logentries

There’s been a lot of industry buzz around Docker recently, with particular focus on its ability to streamline how companies manage their platforms. With all this talk, you might be wondering how easy it is to set up Docker and evaluate it for yourself. While it isn’t too difficult to install Docker directly on a server or virtual machine, Google has recently made it even easier with their cloud platform’s “ Container Engine [https://cloud.google.com/container-engine/]”, now out of beta. Goog

1 min IT Ops

Infographic: Mastering the Shift to Microservices

Microservice architecture is fast becoming a go-to solution for enterprise applications, but it’s not always easy to make the transition from an established, monolithic infrastructure. Lightweight and loosely coupled, building a set of microservices is arguably more difficult than building a monolithic application. However, once established, microservices offer a series of advantages over traditional architectures as deployment times become shorter and iterating becomes easier. For proof of s

1 min IT Ops

Best Practices for Container Log Analysis: Part 1

This 3-part series explores the challenges presented by containers and the advantages of using an end-to-end container log monitoring solution for complete container environment visibility. [http://info.logentries.com/log-analysis-containers] As container environments become mainstream, it’s important to consider the most common challenges of migrating a monolithic application into containerized microservices and how to overcome them. While containers introduce new levels of flexibility from

3 min IT Ops

What is Elastic Logging?

We’re all familiar with the concept of “Elasticity” – the way cloud infrastructures can automatically react to their required workloads, scaling resources up or down as needed. While elastic environments provide us with much needed flexibility, they have also historically presented challenges when trying to monitor activity from their ephemeral components. Automatically accounting for new nodes can be tricky while scaling up. And when scaling down, data associated with these nodes is potentially

1 min IT Ops

Introducing Logentries' Python APM Community Pack

At Logentries, we feel strongly about the power of log data and the unparalleled role that logs can play in effective end-to-end system monitoring. Yet we feel it also important to recognize how other monitoring approaches can further supplement a log monitoring solution to provide even greater, actionable insight into system performance. One such approach is Application Performance Management [/2015/03/terminology-nerd-war-apm-log-analysis-more/] (APM) and today we’re excited to announce our fi

2 min IT Ops

Using Log Data Streams for Real-Time Analytics: Part 2

This 3-part series explores the definition and benefits of using log data streams and real-time analytics for some common IT Ops uses cases. To download the complete article,click here [http://info.logentries.com/real-time-analytics] . [http://info.logentries.com/real-time-analytics] Many analytics tools focus on just one layer of your application stack. Whether it’s Google Analytics tracking events on your website’s front-end, or a server monitoring tool tracking CPU and memory usage, these

2 min IT Ops

Free Docker Logging - Removing Barriers for Mass Innovation

Containerization, led by Docker, is fundamentally changing how we build, manage and monitor systems. When AWS came along they increased the rate of innovation by lowering the cost of failure. This allowed companies (like Logentries) [https://logentries.com] to get started on the back of a credit card with no capital expense up front; and enabled people to easily prove out ideas and concepts in the wild without the risk of having to remortgage their house. Similarly, containers are helping to a

1 min IT Ops

Webinar Recording: Ensuring Docker Success with Real-Time Monitoring

When working with Docker, there’s no one way to monitor your containers and environments. As Docker has evolved from a useful tool for development environments to the industry-changing suite of production-ready services it is today, our ability to monitor containers has also evolved. [http://info.logentries.com/ensuring-docker-success-with-real-time-monitoring] In our latest joint webinar, Logentries Co-Founder Trevor Parsons and DataDog Developer Evangelist Matt Williams discussed the types

14 min IT Ops

State of Log Management for AWS

Introduction The Log Management industry was traditionally driven by regulatory compliance and security concerns resulting in a multi-billion dollar market focused on security and information event management (SIEM) solutions. However, log management has evolved into a market that is focused on both the management and analytics of log data. Log management technologies are becoming more powerful and dynamic, allowing for data to be easily extracted and analyzed from logs for a much wider range

3 min IT Ops

Using Log Aggregation Across Dev & Ops: The Pricing Advantage

Rob Thatcher is Co-founder and Principal Consultant at Skelton Thatcher Consulting [http://skeltonthatcher.com/]. Summary: the pricing of tools or licenses for log aggregation can have a significant effect on organizational culture and the collaboration between Dev and Ops teams. Modern tools for log aggregation (of which Logentries [https://logentries.com]is one example) can be hugely enabling for DevOps approaches to building and operating business-critical software systems. However, the pri

2 min IT Ops

Introducing an Enterprise Logging Solution for Heroku Customers

Heroku is a well-known favorite amongst developers for quick and easy web app deployments that abstract away the hassle of managing and maintaining a hosting environment. Yet over the past year, I’ve heard from an increasing number of large organizations leveraging Heroku for their large-scale applications – whether it be the entire application or specific parts of a larger micro-service environment. Today, Logentries is excited to announce an Enterprise plan for Heroku customers with large en

7 min IT Ops

Using Log Data Streams for Real Time Analytics: 4 Common Use Cases

Introduction Web applications today are part of every IT operation within an organization. Independent software vendors (ISV) as well as enterprises create web applications to support their customers, employees and even suppliers. All in all, the goal of these applications is to deliver a service, complete a transaction, support efficient and effective business operations, and directly impact company revenues and operational margins. In 2009, Paypal went offline for over an hour due to a netw

2 min IT Ops

Optimizing Your Container Environment: Pets vs. Cattle

In the midst of Docker’s meteoric rise and the explosion of talk around containers, it can be easy to lose oneself in all of the new terminology and jargon. While we think about the challenges presented by using containers in production, we also continue to hear the metaphor of Pets vs. Cattle and why it’s important to maintain an infrastructure that acts like a herd of cows. Pets vs. Cattle What is this pets vs. cattle nonsense we keep hearing? Simply put, the “cattle not pets” mantra suggest

3 min IT Ops

How to Add a GPS Time Source to ntpd

USB GPS dongles have come down significantly in price in recent years and I picked one up to play with recently. Apart from using a GPS module to report your latitude, longitude, altitude and time for mapping applications, it’s also possible to feed the time information to ntpd as a back-up time source or as a highly accurate time source depending on the GPS module you end up getting. [/content/images/le-img/2015/07/adding-gps-time-source-to-ntpd.png] The module that I use in this blog post

3 min IT Ops

An All-inclusive Log Monitoring Container for Docker

Over the last year we have watched Docker take the world by storm and when you stop to ask “Why?”, it becomes quite obvious: Docker may eventually replace the hypervisor (see diagram below) with a more lightweight efficient and scalable approach to building tomorrow’s systems. That’s a pretty big deal, so no wonder we are seeing so many organizations moving in this direction and a diverse ecosystem springing up around Docker. [/content/images/le-img/2015/07/containers-vs-Vms.png] At Logentr

2 min IT Ops

Playing with Java 8's Completable Futures

Of the many additions to Java 8 such as the Stream API [https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html] and lambdas [https://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html], I noticed one of the lesser talked about ones was CompletableFutures [https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html] . So I decided to have a play around with them on the last Java component I wrote. My use case in a nutshell was piping larg

4 min IT Ops

Introducing Logentries’ New Querybuilder

We recentlyannounced the release of Logentries Query Language (LEQL [/2015/06/introducing-leql/]), an even more powerful but incredibly easy way to query your log files. The new Querybuilder search tool automatically identifies the available Key Value Pairs in your log events and presents them as options to be included in your query. The Querybuilder also displays a list of available query functions that can be used to calculate values such as COUNT, SUM, AVERAGE, MIN & MAX. [/content/images/

4 min IT Ops

Ditch the Debugger and Use Log Analysis Instead

This guest blog post is written by Matthew Skelton, Co-founder and Principal Consultant at Skelton Thatcher Consulting. -------------------------------------------------------------------------------- Summary: As a Developer, you cannot attach the debugger to your application in Production, but you can use logging in a way that helps you easily diagnose problems in both development AND Production. You also get to make friends with Operations people – win! The applications we’re developing and

1 min IT Ops

10 Best Practices for Log Management & Analytics: Part 1

[http://info.logentries.com/log-management-best-practices] This 3-part series covers Logentries’ 10 best practices for log management and analytics. To download the complete article, click here [https://information.rapid7.com/insightops-10-best-practices-for-log-management-and-analytics.html] . As applications, hosting environments and infrastructure continue to grow in size and complexity, having a well defined set of logging strategies and practices is more important than ever. In Logentri

2 min IT Ops

Using Log Data Streams for Real-Time Analytics: Part 1

This 3-part series explores the definition and benefits of using log data streams and real-time analytics for some common IT Ops uses cases. To download the complete article,click here [http://info.logentries.com/real-time-analytics] . [http://info.logentries.com/real-time-analytics] Analytics tools are often focused on analyzing historical data. Taking a sample of data from historical events, you can perform calculations to determine what happened during that period of time and report on you

6 min IT Ops

How to Implement ANTLR4 Autocomplete

Antlr4 [http://www.antlr.org/] is a new iteration of a popular Antlr parse tree generator. Antlr4 features great documentation [https://theantlrguy.atlassian.net/wiki/display/ANTLR4/ANTLR+4+Documentation] and an in-depth book [https://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference] on the subject. However, the topic of autocompletion lacks any substantive material. I hope this article will steer you in the right direction if you are looking to implement autocomplete functionality

23 min IT Ops

Idiomatic ES6

[](#es6) There are a billion articles on ES6 at this point. What’s one more? Here we discuss some emerging patterns and issues related to real world use of ES6 as well as how one can go about using it now via Babel. If you aren’t yet familiar with the features and changes of ES6 itself, you’ll probably want to check out the following links first:- [MDN](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference) is invaluable. It provides systematic coverage of all JS, including ES6. * ②a

2 min IT Ops

The State of Logging on Docker: What's New with 1.7

Logging on Docker has steadily evolved with each Docker release and there are some exciting new updates that have come with Docker 1.7 – for example, Docker 1.7 now includes new driver support for for Journald [http://www.projectatomic.io/blog/2015/04/logging-docker-container-output-to-journald/] along with the driver support added with Docker 1.6 for JSON and syslog [https://blog.docker.com/2015/04/docker-release-1-6/]! We are here on the ground at DockerCon as part of the Innovators Showcase

2 min IT Ops

Announcing Logentries as Google Cloud Platform's First Log Analytics Partner

Today we’re excited to announce [https://logentries.com/logentries-delivers-first-real-time-log-management-and-analytics-integration-for-google-cloud-platform] our partnership with Google Cloud Platform, making Logentries [https://logentries.com]the first provider of log analytics for Google Cloud customers. [/content/images/le-img/2015/06/logging-from-the-google-cloud-platform.png] Logentries’ Google Cloud integration enables Google customers to perform advanced analysis on their log data,

3 min IT Ops

Introducing Logentries NEW Query Language: LEQL

We are excited to announce that Logentries’ new SQL-like query language, LEQL, is now available [https://logentries.com/logentries-new-analytics-language-makes-the-power-of-log-data-accessible-to-the-masses/] for more advanced analytics and easy extraction of valuable insights from your log data. A SQL-Like Query Language If you’ve ever used SQL, LEQL should feel familiar. In fact, Logentries already supports a number of SQL-like search functions, including: * SUM: Sums a set of values *

2 min IT Ops

How to Log with the Docker Logentries Container

Logentries offers a variety of ways to get logs out of your containerized environment [/2015/04/logging-on-docker-what-you-need-to-know/], including our Linux Agent, application plugin libraries, and Syslog. In this post we’ll cover collecting and forwarding logs via our Docker Logentries Container, which requires Docker 1.5 or higher. To configure the Docker Logentries Container you’ll need to do the following: * Create a destination log in your Logentries account to record your Docker lo

1 min IT Ops

Migrating to Micro-services with CoreOS & Logentries

It’s no secret that container-based architectures have become an extremely popular choice amongst organizations looking to simplify and automate their deployments. One of the leaders of this movement, along with Docker, is CoreOS providing teams with the ability to manage their services through neatly packaged Linux containers. [http://info.logentries.com/containers-and-microservices-recording] It’s always great to move to a new architecture with a new project, but what about existing servic

2 min IT Ops

A Beginner’s Guide to the DevOps Tool Arena—and Webinar!

Guest author Asami Novak is a content strategy and development manager at New Relic [http://newrelic.com/]. (This post is adapted from the original on the New Relic blog [https://blog.newrelic.com/2014/11/10/devops-tool-arena/]. For more tips and advice to help get your DevOps practice in gear, check out New Relic’s free “ Kickstarting DevOps [https://try.newrelic.com/rs/newrelic/images/NewRelic-Kickstarting-Devops-eBook.pdf] ” ebook.) Whether you’re practicing DevOps today or gearing up for D

4 min IT Ops

Migrating to Bootstrap 3 with Legacy Code

Bootstrap 3 is now established and stable after its release a year and a half ago, but there is still a sizeable number of websites and web applications that are still styled using Bootstrap 2. If you’re starting to build a web application, especially for quick prototyping, using Bootstrap 3 is the clear option. But, if you already have a mature application, it is likely you have a large amount of code that needs to be refactored. This refactoring may not be easy as there are a lot of chang

2 min IT Ops

Enabling DataOps with Easy Log Analytics

DataOps is becoming an important consideration for organizations. Why? Well, DataOps is about making sure data is collected, analyzed, and available across the company – i.e. Ops insight for your decision-making systems like Hubspot, Tableau, Salesforce and more. Such systems are key to day-to-day operations and in many cases are as important as keeping your customer facing systems up and running. If you think about it, today every online business is a data driven business! Everyone is account

4 min IT Ops

Integrating etckeeper with Logentries & Chef

When working within a team to maintain system infrastructure, properly documenting and communicating changes made to configuration files within /etc is fundamental to preventing knowledge gaps throughout your team. [/content/images/le-img/2015/05/integrating-etckeeper-and-chef-with-le-1.png] While version control tools like git are helpful in tracking standard changes to a code base, git doesn’t capture metadata important to /etc like permissions of /etc/shadow. To address this need, we’ve b

0 min IT Ops

#VelocityConf 2015: Don't-Miss Speakers and Sessions

[Velocity 2015 is kicking off tomorrow May 26th with an exciting line-up of events and presentations from some of the best web ops and performance leaders, do-ers, and visionaries. We are looking forward to some of these great speakers, panelists and sessions!](//storify.com/trevparsons/looking-forward-to-velocityconf-2015-speakers-and-)]

2 min IT Ops

Delivering Reliability with Logentries and Microsoft Azure

Today, we’re announcing our partnership with Microsoft Azure, making it easy to centralize and analyze log data from VMs and web apps hosted on Azure. One of the things we love about Azure is how easy it is to spin up new services quickly. I’ve found that the operational flexibility Azure enables through these out-of-the-box solutions is particularly helpful to organizations using distributed systems that span solutions located both on-premises and in the cloud. [/content/images/le-img/2015/05

2 min IT Ops

Making SIEM Easier to Achieve

A June 2014 Gartner Report [http://securityintelligence.com/gartner-2014-magic-quadrant-siem-security/#.VVtFRtNVhBc] on Security Information and Event Management (SIEM) reveals that many surveyed SIEM users indicate “cost” (both in terms of price and effort) as being one of the biggest challenges presented by traditional SIEM tools of today. The irony of this insight is that reducing the cost and complexity of managing security information and events should be the primary function of a SIEM too

0 min IT Ops

Getting Ready for #VelocityConf 2015? Let's Recap 2014!

The Logentries team is on its way to Santa Clara for Velocity 2015 in just a few weeks. Here are some of the highlights from last year’s #VelocityConf[](//storify.com/trevparsons/recap-velocityconf-2014)

4 min IT Ops

Fun with JavaScript on the JVM

It’s easy to see how JavaScript is everywhere these days. The barrier to entry is extremely low; anybody with a browser can write and evaluate it, and with advancements in runtimes like Google’s V8 [https://code.google.com/p/v8/], writing server-side JS is now a viable proposition. It’s easy to forget, then, that Rhino- one oforiginal JavaScript interpreters was written in Java. Not only that, but Mozilla is still looking after [https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino/His

4 min IT Ops

Metric-driven Smart Deploys

Automated deployment isn’t just a wonderful thing — it’s a necessity when it comes to providing consistent, error-free delivery without eating up all of your team’s time and resources. You create a set of scripts to cover standard, known deployment cases (including the most likely contingencies), then stand back and let them do all of the heavy lifting. It doesn’t just save time compared to manual deployment — it also reduces errors, while at the same time providing you with a perfect brea

3 min IT Ops

Building Tabular Data Format with D3

In my previous posts I have demonstrated how to combine D3 with AngularJs [/2014/08/how-d3-can-help-you-build-effective-data-visualizations/] (wrapping D3 code in Angular’s directives) and how to build effective data visualization with D3. In this post I will focus on building tabular format of data with D3. [/content/images/le-img/2015/05/building-tabular-data-format-with-d3.png] Tabular representation of data is often the simplest and clearest way of showing the data to the user. Graphical

3 min IT Ops

What You Can Learn From Log-level Usage Reports

Separating the signal from the noise is one of the biggest challenges when dealing with machine-generated log data today and has generally required deep technical expertise. However once you find that signal it can be massively useful and can help you make business decisions with a big impact. Today Logentries is announcing log level usage reports [http://info.logentries.com/Log-Analysis-Report-Sign-up], which is one more way we are striving to do the hard work so you don’t have to! [/content/

2 min IT Ops

Listening to What Your Heroku Postgres Logs are Telling You

Debugging in development can be challenging enough when dealing with just your application layer. In production, the root cause of an issue could be hiding within one of several locations, including your database. [/content/images/le-img/2015/05/Heroku-Postgres.png] One of the many great things about Heroku [https://logentries.com/doc/heroku/] is how well it incorporates different types of events into its own logs. When tailing Heroku logs, you can see events from not only Heroku, but also

4 min IT Ops

DataOps: Creating a Culture of Data Analysts

Bill Saltmarsh is a Logentries Customer and Analytics Engineer at Pluralsight. [http://www.pluralsight.com/] Today’s rapidly evolving workplace provides a context where our decision makers must be able to leverage their data to make optimal judgments. This is not limited to managers and organizational leaders. On the Contrary, We Are (or soon will be) All Data-driven Decision Makers in Our Organizations. [/content/images/le-img/2015/05/dataops-creating-a-culture-of-data-analysts.png] The is

3 min IT Ops

eComm: Living & Dying by Transactions

Success or failure of e-Commerce sites boil down to transactions. In brick and mortar stores, transactions are handled by point of sale systems. Their operation is outsourced, and their complexity is low. But for e-Commerce applications, the transactions not only rely on the processing API you choose, they also rely on the servers, the performance of the web application, the user experience and more. And your ability to adapt to the users in real-time, is what helps you drive a repeat visit and

4 min IT Ops

Salt-ssh for Remote Execution of States and Modules

This article assumes that the reader has a basic understanding of how Salt works and regular Salt usage experience with master and minions. The goal of this post is to show how to use Salt-ssh in a non-root environment without a master or minion. [/content/images/le-img/2015/04/salt-ssh-for-remote-execution-of-states-and-modules.png] Why Would You Use Salt-ssh to Execute States and Modules? * If you already have invested time and effort in a SaltStack installation and you have some machine

1 min IT Ops

Announcing Logentries User Access Controls

Sending your logs to a centralized location [https://logentries.com/insights/log-data-centralization/] like Logentries creates the opportunity for different teams to access valuable log data without granting everyone access to your production environments. While this exposure of more teams (both technical and non-technical) to log data promotes collaboration across your organization, it can also leave you potentially vulnerable to the possibility of team members accidentally changing or deletin

2 min IT Ops

What is DataOps & Why You Need It?

Mike Roberts is a Logentries customer and Director of Data Analytics at Pluralsight [http://www.pluralsight.com/]. In the world of technology, things change. We all know this so don’t feel there’s a huge need to convince people of the fact. Often times, there are buzz words which surface and catch on like a rampant wildfire: there’s no escaping it. Some of these continue on and become verifiable domains, and others wither and are quickly extinguished. [/content/images/le-img/2015/04/what-is-

3 min IT Ops

Log Analysis for Orchestration Change Management

Are you suffering from server sprawl? You might be and don’t even know it. Server sprawl occurs when there are an unknown number of rogue VMs; VMs with unknown workload, owners, or purpose. And no one is fearless enough to delete or suspend them. Orchestration tools make creating new nodes so easy that almost every organization who uses them is suffering from server sprawl…at a high price and lots of confusion. [/content/images/le-img/2015/04/log-analysis-for-orchestration-change-management-

4 min IT Ops

10 Ways to Lead with Analytics

Today, everything must be faster. Faster releases; faster and larger backlogs; faster complaints; faster bugs. The speed and performance of the modern software delivery pipeline brings a new set of challenges. How do you keep up? There really is only one option — automation and measurement. You need a method for automatically collecting data about your application and how it is being used. But also a good way to find the relevant information and act on it. And in order to do this your developmen

1 min IT Ops

Java Util Logging - Now Available!

Just a quick update that Logentries now supports Java Util Logging. Thanks to GitHub user raupachz [https://github.com/raupachz] you can now use java.util.logging to send logs to Logentries [https://logentries.com] using the open source le_java [https://github.com/logentries/le_java] library. [/content/images/le-img/2015/04/java-util-logging.png] You can also use log4j [https://github.com/logentries/le_java/wiki/Log4j], log4j2 [https://github.com/logentries/le_java/wiki/Log4j2] or logback [h

4 min IT Ops

Introduction to Apache Spark

Apache Spark is a fast and general-purpose cluster computing system. The latest version can be downloaded from http://spark.apache.org/downloads.html. In this post, we will try to perform some basic data manipulations using spark and python. Using Spark Python Shell The first thing a Spark program should do is create a SparkContext object, this tells Spark how to access a cluster. When you use the python shell, a context variable named “sc” will be created automatically. To access a python S

3 min IT Ops

DataOps: How Pluralsight Uses Tableau & Logentries for Better Analytics

*Mike Roberts is a Logentries customer and Director of Data Analytics at Pluralsight. * A truly stable and robust Analytics platform is able to support both the analysis of external data *as well as *internal data, or data about its own state. Tableau Software is one such analytics platform and Logentries [https://logentries.com] is the ‘listener’ that makes analyzing Tableau’s system data easier. As a result, our Pluralsight [http://www.pluralsight.com/]team is better able to understand the fi

1 min IT Ops

Logging on Docker: What You Need to Know

Systems have fundamentally changed with the introduction and adoption of micro services like Docker, resulting in a shift in how we think about log management and analysis [https://logentries.com]. Logging into a server and grepping logs is no longer a reality when dealing with thousands of container instances. While support of logging has improved on Docker over the past year, there is yet to be a widely agreed upon “standard” for Docker logging. [/content/images/le-img/2015/04/logging-on-doc

4 min IT Ops

The Hidden Costs of Open Source

Open source projects allow us to adopt technology without a lot of hoops. We get to leverage a strong community base, and potentially support the growing group committed to building even better solutions, which can have the added benefit of notoriety. But there is one problem – open source is not as it seems. While it might appear free to begin with, it does come with cost down the road in real dollars, pain, risk, and time. Many companies are open-source first. Where every new challenge is

2 min IT Ops

How to Track Meteor's Recomputations with Logentries

This blog post is written by Logentries customer Gabriel Pugliese, Founder and developer of CodersTV [http://coderstv.com/]. There are a lot of implicit computation going on with Meteor’s reactivity. Tracker, Session variables, Cursors, Templates etc. They create lots of implicit recalculations that we can easily lose their tails and create a seven-headed dragon that will haunt you! For me, the immediate valuable technique is logging everything that is going to recompute on reactive data contex

7 min Logentries

The Flexbox Paradigm: CSS3 Layout for Today’s Applications

Introduction Controlling the layout of web pages and applications has always been a little tricky. In the beginning, there were almost no mechanisms for page layout, other than some basic formatting of html tags. We could apply some font styling, add background colors, and with the use of paragraph’s and line breaks could achieve some block spacing. With the introduction and evolution of CSS, it gave us further control, but more importantly, control over the elements box model. We could now f

4 min Logentries

MongoDB Log Analytics

MongoDB 3.0 is now available! If you are new to MongoDB or upgrading from 2.6, you will enjoy all of the new features including document-level locking, better write performance, big memory support, and more. Additionally, to improve usability of the log messages for diagnosis, MongoDB now categorizes some log messages under specific components, operations, and provides the ability to set the verbosity level for these components. Today, Logentries is launching a new Community Pack for MongoDB

2 min IT Ops

Instant Log Gratification

Sometimes you just don’t want to wait. For example, most DevOps teams don’t want to talk to a sales guy before they try out a new product, and don’t want to go through a lengthy or painful sign-up process. In fact I get frustrated if I even have to install or configure something in order to get an understanding for the value of a given tool or solution.This is especially true when you want to try something out for evaluation purposes, or want an immediate answer/insight into what is happening in

4 min IT Ops

PaaSt Times

Boy was I naive. When I first learned about the term PaaS, I threw it away quickly into a metaphorical garbage can. Of course you need your servers I thought. This was about five years ago. But now, I realize how silly that was. While IaaS is still the primary backend for applications, there are many out there that contain a combination of PaaS and IaaS, and a rare handful that are 100% PaaS. And for both of these scenarios the importance of log management [https://logentries.com/doc/log-manage

4 min IT Ops

Getting Started with DevOps

John-Daniel Trask is the Co-founder and CEO of Mindscape, creator ofRaygun [https://raygun.io/], an error tracking tool that provides real-time tracking of software bugs and crashes that are happening in your applications. You’re launching a new application. It’s exciting and you can see the light at the end of the tunnel! Releasing a product sometimes feels like the end of a journey, but it’s actually like considering birth the end of a journey – it’s not, it’s the start of something much bigg

2 min IT Ops

New Logentries Cookbook for Chef

[/content/images/le-img/2015/03/chef-and-logentries.png#img=half-right]We have released our logentries_agent cookbook to supermarket.chef.io [https://supermarket.chef.io/cookbooks/logentries_agent]! You can check out the docs here, [https://supermarket.chef.io/cookbooks/logentries_agent]or I’ve developed the following brief tutorial to walk you through how to automate your installation of the Logentries Linux Agent [https://logentries.com/doc/agent/] in your own infrastructure. First off, I

4 min IT Ops

Terminology Nerd War: APM, Log Analysis & More

Just the other day I was hanging out with my developer buddy. We entered what we thought would be an interesting topic on how you cannot call an environment “DevOps” without analytics. But we soon were in a nerd war on what a term meant. Yes, this is what I talk about in my free time. [/content/images/le-img/2015/03/nerd-war.png] In the thick of it, we both used the term “Server Monitoring.” But neither of us were talking about the same thing. I was referring to log management [https://logentr

2 min IT Ops

How to Centralize Logs from CoreOS Clusters

Containerization and microservice architectures are commonly resulting in highly distributed systems with large numbers of dynamic and ephemeral instances that autoscale to meet demands on system load. It’s not uncommon to see clusters of thousands of container instances, where once there were tens of physical servers, now there are hundreds of (cloud) server instances. Because containers are extremely lightweight, they allow code to run in isolation from other containers while safely sharing t

3 min IT Ops

Has Splunk Turned Its Back on the DevOps Community?

In August 2012 Splunk released Splunk Storm [http://www.splunk.com/view/SP-CAAAG7S], tailored for “organizations that develop and run their applications in the public cloud, using services such as Amazon Web Services (AWS), Heroku, Google App Engine, Rackspace, and others.” Splunk Storm claimed to provide “developers” with an easy-to-use, subscription version of the Splunk software. Since this grand launch of Splunk Storm, to meet the needs of the DevOps community, Splunk has decided to end of l

5 min IT Ops

Template Caching in Angular - How to Avoid the Storm?

When we build a JS application, we usually start thinking about optimizing the http request, compression and, sometimes, “uglifying” the JS files. [/content/images/le-img/2015/03/template-caching-in-angular-how-to-avoid-the-storm.png] But, what about the html files? Let’s see if we can optimize the number of requests needed to load the templates for our own controllers and directives! Using Templates Without Impacting Your Server For a start, here are the two best ways that I know to associa

3 min IT Ops

Why Interoperability is a Key Requirement for Your DevOps Toolkit

Today’s DevOps culture drives the requirement for development and ops teams to share tooling and to combine lots of different services/processes to give them visibility into their systems throughout the system life cycle. For example the modern DevOps toolkit [/2014/12/infographic-the-modern-it-and-dev-ops-toolkit/] will consistently include the following categories of tools (and more…): * APM (e.g. New Relic, AppDynamics, AppNeta, AlertSite) * Team Communication (e.g. Slack [https://logen

3 min IT Ops

Getting Started with the Logentries & Logstash Integration

Logstash is an open source tool for managing events and logs. It is used to collect, search and store logs for later use. If you are using Logstash to collect logs from across your infrastructure already, and you are looking for more sophisticated log analytics tool, you are in the right place. I will show you how to configure Logstash to forward all your logs to your Logentries [https://logentries.com]account using the plugin and token connection. Prerequisites * Logstash downloaded and co

3 min IT Ops

Logging Your Entire Software Delivery Pipeline

When we think of traditional development and production operations, we often put everything into a linear software delivery pipeline that starts with a development backlog, and ends with production monitoring. We slot tools at each stage, and for the most part, keep everything segmented. Log analysis [http://logentries.com]is a common tool in that chain but where does it fit? At the end? I think not. Log analysis can be used throughout your entire software delivery pipeline. The linear pipeline

4 min IT Ops

Acceptance Tests In Practice - Behavior Driven Development

What is Acceptance Testing? "Acceptance testing is a test conducted to determine if the requirements of a specification or contract are met.” (Wikipedia definition [http://en.wikipedia.org/wiki/Acceptance_testing]) In simple words, Acceptance tests check if the software that we have built matches the requirements that were provided. The Magical Black Box Acceptance testing is usually performed using “black box [http://en.wikipedia.org/wiki/Black_box]” testing method. The tester of the system k

4 min IT Ops

The Role of Log Analysis in Our Technical Transformation

Hello, Logentries readers – I’m the VP of Technology at Motus [http://motus.com] , a SaaS company for mobile workforce management, headquartered in Boston. Motus has undergone a huge technical transformation in the last 18 months, and logging has been a big part of that transformation. I wanted to share with you some of our experiences and where we see the future of log management [https://logentries.com/doc/log-management/] and analysis going here at Motus. Where We Started… 18 months ago,

3 min IT Ops

Installing Logentries Using Ansible

*This post was written by Logentries [http://logentries.com] customer Richard van den Brand, Software Engineer at Waarneembemiddeling.nl, where he is responsible for developing and maintaining their Symfony2 applications and infrastructure. * In this short tutorial I’ll describe the steps required to install the Logentries agent using Ansible. This tutorial assumes you have some basic knowledge about Ansible and how to use playbooks and roles. If you’re unfamiliar with these topics please co

3 min IT Ops

What Is the Docker Stats API?

Containerization and micro-services are changing how development and operations teams design, build and monitor systems. Containerization of environments regularly results in systems with large numbers of dynamic and ephemeral instances that autoscale to meet demands on system load. In fact, it’s not uncommon to see thousands of container instances, where once there were hundreds of (cloud) server instances, where once there were tens of physical servers, From a monitoring perspective this mean

4 min IT Ops

Machine Data for End-to-End IoT System Monitoring

The Internet of Things (IoT) is really the morphing of physical devices into a software world. This means a lot of the same things you would do with applications and servers, you can now do with devices, including leveraging data to respond faster to issues, track trends, and improve your roadmap. The Internet of Things, Internet of Everything, Machine-to-Machine (M2M), Smart Systems etc. all refer to devices or gadgets that are intelligently connected to their users and the web, in order to

4 min IT Ops

4 Steps for Structuring Your Log Data

In the age of BigData we are taught that no pile of data is too large or too complex. This is absolutely true. Most data analysis systems can take any type and volume of data — but ingestion is much different from consumption. The way your data is structured directly impacts its ability to be consumed, understood, and correlated with other data. Here are the top 4 ways to make sure your system and app logs help you to do this effectively. Before you start ingesting a new source type, spend a li

2 min IT Ops

Logging From Your Java Application Using log4j2

In a previous post [/2014/06/sending-logs-from-your-java-application/] I described how you can log from your java applications to Logentries [https://logentries.com] using log4j and logback with our open source le_java repository on github [http://github.com/logentries/le_java]. Well now thanks to github user joshuadavis [http://github.com/joshuadavis] you can now log from log4j2 too! See our product documentation on how to get started with le_java and log4j2 [https://logentries.com/doc/java/

2 min IT Ops

Ubuntu & Logs at the Core of IoT

I always think of Internet of Things as a world of devices with (wireless) internet connections all communicating with each other. All such devices spew out logs (as all software and hardware components do) as they run – akin to exhaust fumes from your automobile (before it was an electric one of course…). However the log data is not simply waste, as is the case of your car fumes, and in fact can act as both telemetry data for monitoring purposes as well as an intermediate language for intercom

3 min IT Ops

Log Analysis for Software-defined Data Centers

Modern infrastructure constantly generates log data at a rate faster than humans can easily analyze. And now that data centers can be built and torn down with scripts, the amount of activity and data is exponential. The traditional log analysis practices of manually reviewing log files on a weekly or daily basis, remain inadequate when looking at Software-defined Data Centers (SDDC). The modern architecture of SDDC, with its highly automated and dynamic deployment capabilities for multi-tier ap

3 min IT Ops

The Role of Logging in ChatOps

What is ChatOps you might ask? Well, if you are like me, you and your team may be already be doing it and you don’t even know it. On a recent visit to San Francisco I was telling one of our tech partners about our new Slack integration [https://logentries.com/logentries-slack-integration-powers-faster-collaboration-real-time-resolution-dev-ops/] when he responded by saying, “Slack and Logentries, cool! Yep everyone is doing ChatOps these days…” So what is ChatOps? ChatOps is an ultra collabo

3 min IT Ops

The Value of Easy: You Be the Judge

Being the giant in the market means you can step on people if you don’t like what they say, how they say it, or if they indicate maybe, just maybe, they have a competitive solution or an opinion that that you don’t like. We saw this happen last week when Chris Riley, a friend of Logentries and an active Devops practitioner, shared his opinion [http://www.hoardinginformation.com/6-reasons-why-splunk-might-be-bad-for-you/] on how to approach log analysis implementation and why some more curren

3 min IT Ops

Infographic: Top 4 Log Management Challenges

At Logentries [http://logentries.com] we chat to new users everyday who are looking for an improved solution for centralizing and analyzing their log data. They have often tried rolling their own solution, have previously gone the open source route, or are using an “old school” logging technology. But, what we find across new users, regardless of how they are managing their log data, are some common challenges that have historically made log management [https://logentries.com/doc/log-management

1 min IT Ops

Friends Don't Let Friends Dig Through Data

Traditionally, log management and analytics has not been easy. We talk to customers who have spent hours digging through logs, writing complex search queries, and waiting…and waiting… for results to populate.We know there is a better way. Logentries [http://logentries.com] offers a free account [http://logentries.com/get-started/] to anyone who wants to see for themselves that log management and analytics can be easy, fast, and incredibly valuable to your IT, DevOps, and business analytics. And

4 min IT Ops

Unit testing with Karma and Jasmine for AngularJS

So you’ve started to build applications with AngularJS; or, maybe you’ve done unit testing before but haven’t used it with AngularJS; or, maybe it’s all new to you, but either way this article should help to orient you to the value of using Karma with Jasmine and offer some tips on displaying functionalities of Angular JS. Let’s get started! My app is small it doesn’t need any tests, right? [https://logentries.com/centralize-log-data/] Wrong! A small app can have errors and it’s bad practic

2 min IT Ops

Gathering Process Metrics on the Windows Agent

As we continue to expand Logentries [http://logentries.com]’ powerful search and metric capabilities, we are also looking for new ways to allow our Users to send us critical data. In the newest version of the Windows Agent [http://rep.logentries.com/windows/Windows-Agent.zip]users can now follow process metrics and send them to Logentries. This feature enables even more insight into your application by allowing you to monitor your processes resource allocation in Logentries. Additionally, with L

4 min IT Ops

Dynamite Plots, Logs, & the Joy In Knowing

I saw this online and chuckled. I believe it was Mark Twain that said, “Humor is the good natured side of truth.” Well, in my humble opinion, humor can be used as the genesis for interesting blog posts. So, you may be wondering, what do Dynamite Plots, logs and the joy of knowledge have to do with each other? Well, if you have a few minutes, I’ll try to make the link. For me, the humor of the cartoon above is rooted in multiple areas, but most of all in the idea that the data inherently has

5 min IT Ops

Fastly Community Pack: Understanding User Experience with Log Analysis

Once you learn what the normal baseline on your network is, you should start to look for abnormal activities. This will help you understand any sudden changes that might affect the overall performance and help in easier troubleshooting. If you are already caching with Fastly [http://www.fastly.com], you might seek to pull more information from your log data to give you deeper visibility. There are a couple of formats in which you can send logs from Fastly to Logentries. You get a choice of eithe

4 min IT Ops

Using Regular Expressions to understand NginX, Apache and IIS logs

Regular expressions offer something that automata do not: a declarative way to express the strings we want to accept. This is why we use it as the input language for our platform to process logs in many heterogeneous formats. When we learn how to extract key values using RegEx from any log format [/2015/01/extracting-key-values-from-any-log-format-using-regex/]we can start to think how to apply it to some of the more popular log formats. In this blog post we take the most popular log formats fo

6 min IT Ops

Extracting Key Values from Any Log Format Using Regex Named Capture Groups

Logs are event streams that are constantly spewed from every application, server instance, mobile and IOT device. They contain valuable information pertaining to application errors, system performance, security, feature usage and more. Likely, the biggest issue with getting value from log data has been its structure, or lack of it… and while today many systems can produce JSON structured log data with keys and values that can be easily identified and analyzed by both human and machine, the vast

3 min IT Ops

Catching Inactivity Before It Catches You

What does not occur is just as important as what did. And unfortunately because IT Operations is usually just one step behind the activity of production, it often forces us to only consider what happened, and sometimes neglecting or missing what didn’t. The impact of a missed script run, or system update, has a ripple effect, and is hard to catch. This is why inactivity monitoring [https://logentries.com/product/inactivity-alerting/]is so important. Typical event monitoring only has value when

5 min IT Ops

Carefree, Scalable Infrastructure

There is a whole lot of talk about this DevOps [/2014/10/6-challenges-facing-devops-and-operations-teams-in-2015/] thing. Pushing teams to move faster, increasing focus on results, and doing so with better and better quality. But the elephant in the room is how we go from immutable infrastructure to scalable environments that move with the code, not against it. And making infrastructure move at the speed of code, takes more than orchestration tools. Operations needs to be confident that they can

3 min IT Ops

Killer Data Center Apps

Wow, this cloud OS scene is really speeding up. Without going all Ray Kurweil [http://www.ted.com/talks/ray_kurzweil_on_how_technology_will_transform_us] on you, I can’t keep up – literally! In my last blog post, I said I’d cover Kubernetes in more detail, explore what plans Mesosphere might have for Mesos and explore how Linux containers would play with clustering. In the intervening six weeks, Google launched the Kubernetes-based Google Container Engine, so I started writing about that. But t

3 min IT Ops

Connected QA: Selenium + Log Analysis

Does the idea of adding Quality Assurance (QA) test runs to your log analysis platform sound strange? It’s actually not! The value in adding QA test runs goes beyond helping the QA team; it helps the entire development shop as they move to a DevOps framework. [/content/images/le-img/2014/12/Using-Log-Analyis-for-Selenium-QA-Testing.png] I talk a lot about the integrated development environment and going beyond ad-hoc tool selection to a more deliberate and holistic approach. This also means br

4 min IT Ops

Is Server Monitoring Dead?

Once upon a time one of the first pieces of software you installed on your infrastructure was a server monitoring tool. This was the control panel through which you could view how effectively your infrastructure was being used, akin to the speedometer and temperature gauges on your car, or the dashboard of dials in an airplane’s cockpit. Server monitoring tools usually work by capturing resource usage information from your OS performance API or performance counters (e.g. via proc stats on linu

6 min IT Ops

End-to-End IoT Monitoring with Log Data

A recent blog [/2014/11/what-is-the-role-of-logging-in-the-internet-of-things] explored the importance of logging in general in terms of IoT devices. It also cited predictions that a huge number (50 billion) of IoT devices are expected to exist by 2020. While Machine to Machine(M2M) communication is a related area, the IoT is all about extending the Internet to systems and even low power devices. While there is uncertainty about the exact nature of how these devices will be networked and how th

4 min IT Ops

Best Practices for Cloud Logging, Security, & Data Protection

When we first founded Logentries in 2010 a lot of people thought Viliam Holub (co-founder, CTO, and the brain behind processing billions and billions of log events in real time [https://logentries.com/real-time-log-stream-analytics/]) and I were crazy. The common response was: “People are not going to send their logs to the cloud… logs might contain very sensitive data…” Like typical stubborn founders we persevered in spite of this, and today we have more than 35,000 users across 100 countries

3 min IT Ops

Community Packs for AWS: Out of the Box Dashboards, Alerts, & Queries

We recently released Logentries Community Packs [https://logentries.com/logentries-launches-shareable-community-driven-log-analytics/] , dynamic JSON files that (when uploaded into Logentries [http://logentries.com] ) automatically create Saved queries, Dashboards and Alerts. The true power of these packs is that anyone can create, modify and share them. We called them Community Packs because we want to offer different Communities a “pack” of log analysis features customized for their specifi

5 min IT Ops

Keepalived and HAProxy in AWS: An Exploratory Guide

We’re going to explore high availability and load balancing using Keepalived [http://www.keepalived.org/] and HAProxy [http://www.haproxy.org/]. Keepalived is a routing software designed to provide simple and robust facilities for load balancing and high-availability to Linux systems and Linux-based infrastructures. HAProxy is an open source load balancer/reverse proxy generally used for load balancing web services, but also has the functionality to load balance TCP traffic. Together, Keepali

6 min IT Ops

Infographic: The Modern IT and Dev Ops Toolkit

Over the past year I reckon I have spoken to more than a thousand Developers/IT Os/DevOps folk through customer calls, demos of Logentries, at conferences such as Velocity, DevOpsDays, AWS re:Invent as well as a bunch of other more low key meetups across US and Europe. Naturally, one of the first questions I tend to ask is: “hey what do you use for logging?” Quickly followed by: “What other tools do you use?” Below is a list of tools I frequently come across (note: this is not exhaustive) that

6 min IT Ops

ES6: Javascript in Harmony: An Overview of New Language Features

ECMAScript 6 brings powerful new capabilities and some tasty syntactical sugar to the ubiquitous Javascript language, as it continues to grab an ever increasing slice of developer mindshare. Now that the ES6 feature set is frozen and just minor refinements will be made over the coming months, much of this new functionality has already landed in modern browsers, and will continue to roll out with each new browser update. For the impatient, there are also now several excellent precompile / polyf

4 min IT Ops

Is Your Log Data Presentation Worthy?

You’re in a quarterly meeting where everyone is armed with slide decks; some good, some mind numbingly bad. It’s your turn, and you have a concise five-slide deck comprised of targeted, well designed, and insightful graphs. The graphs are clear and the room is able to discuss the highs and lows reported in the data. While you leave the meeting satisfied that you communicated clearly… The complex data you turned into consumable information was no trivial task. Regardless if it took a few hours

3 min IT Ops

How to Implement JSON Formatting of IIS for Analytics and Troubleshooting

Previously, we wrote about setting up a Windows environment [/2014/11/json-formatting-of-windows-events-its-hot/] to log JSON formatted logs using our Windows Agent or our DataHub. Now we’ll tackle IIS [http://www.iis.net]. IIS, like Windows, has a unique log format that makes it difficult to read, parse, and garner useful information. The log is a flat file that has a line-per-web hit; similar to Apache or Nginx. But, it’s not as easy to format intoJSON as Apache and Nginix [/2014/08/json-lo

4 min IT Ops

For the Love of Code: Why We Use JSHint for Static Code Analysis

As developers, we all strive for clean, readable, and easy to refactor code; but, unfortunately, this doesn’t always happen. No matter how great a developer you are, or what language you’re coding in; problems caused by bugs inevitably spring up like weeds in the grass. These problems are exasperated by poorly organized and poorly written code. Once quality starts to drop, even the cleanest high-quality code in your project begins to be affected, until you’re left with a jumbled mess of (and ha

3 min IT Ops

How to Adopt DevOps in Your Organization

It does not take much to understand the benefits of the DevOps culture, processes, and tools. However, implementing DevOps in your organization is not as obvious and usually involves more than simply setting up tools. You have to convince team members, map old processes to new, and maybe even change the structure of organizational reporting and budgeting. Unfortunately, there is no magic formula for implementing DevOps in an organization, but there are some strategies to help. [ One proven

2 min IT Ops

An Introduction to Logentries Community Packs

We have built our new Community Pack technology as a way for small or large communities to share ways to extract the important information generated by their systems and technology. What are Community Packs? It’s a completely open service allowing people to create packs containing saved searches, alerts, tags, and dashboards which use the Logentries infrastructure and tools, but focused for the technology used by their community. We have also created a new repository at https://community.log

3 min IT Ops

JSON Formatting of Windows Events: It's Hot!

It seems like everyone in DevOps has been talking about JSON recently – JSON is hot! Logentries has written a few posts covering this topic, coveringWhat is JSON [/2014/11/what-is-json-an-introductory-guide/], Common Problems Solved with JSON [/2014/09/3-common-problems-solved-with-json/], and our Exporting as JSON [/2014/07/logs-to-metrics-to-json-to-geckoboard/]. However we thought it would be beneficial to dive into some more specific applications. We already wrote about a few, namely JSON a

4 min IT Ops

Unlimited Logging: A New Chapter in Log Management

It’s no secret that log data is quickly becoming one of the most valuable sources of information within organizations. There are open source, on-premise, and cloud-based solutions to help you glean value from your logs in many different ways. Largely, organizations use logs for debugging during development, for monitoring and troubleshooting production systems, for security audit trails and forensics, and (more and more) for different business use cases that transcend product management and mar

3 min IT Ops

The Role of Logging in the Internet of Things

Let’s explore the importance of logging from IoT devices… The IoT is defined as the interconnection of uniquely identifiable embedded computing devices within an existing internet infrastructure [1]. In plain English IoT is expected to enable the advanced connectivity of devices and systems, which would involve machine-to-machine communications (M2M). The IoT [http://en.wikipedia.org/wiki/Internet_of_Things] is still in its early stages but some analysts predict that the IoT will boost the glob

4 min IT Ops

What Is JSON? An Introductory Guide

Some days it’s hard to remember if Moore’s law applies to increasing computer power or the number of technologies and breadth of terminology impacting our daily work. JSON, short for JavaScript Object Notation, continues to gain momentum in our increasingly connected world. Reading this primer will give you a baseline understanding so you can start having intelligent conversations about the pros and cons of using JSON [/2014/10/how-to-format-nginx-and-django-logs-using-kvps/] with your team.

5 min IT Ops

4 Ways to Ensure That Continuous Delivery Helps, Not Hurts

Customer demands aren’t the only thing pushing development and operations teams into more frequent software releases. It is also the need for quicker feedback on product quality, the desire to reduce bottlenecks in operations teams, and the goal to utilize less overhead on projects. The concepts and spirit of continuous delivery [/2014/07/smart-continuous-delivery-using-automated-testing/] are well known. However, organizations with existing applications starting to implement continuous deliver

3 min IT Ops

Logging Activity in a Smart Home

The Smart Home concept is a subset of the Internet of Things(IoT). The core idea is to connect “things” (digital devices) to each other to facilitate communication, feedback, and alerting. In essence connecting the physical world with the digital world. We are installing new sensors and actuators into everyday devices that is leading to new IoT and Smart Home services by integrating existing solutions and technologies. The IoT network is growing at an unbelievable pace. From just 2 billion obje

3 min IT Ops

Mesos, Yet Another Cloud OS?

YARN stands for Yet Another Resource Negotiator. According to Hortonworks [http://hortonworks.com/hadoop/yarn/], it is “the architectural center of Hadoop.” YARN is the stack layer that allows multiple processing engines, with varying workloads (streaming, SQL, batch, machine learning etc.) to run on top of the Hadoop file system, HDFS. Before YARN there was another resource negotiator, Mesos (Hence the “yet another” in YARN) Mesos was originally documented [https://www.cs.berkeley.edu/~alig/p

4 min IT Ops

How to Use Node-Red with Logentries for Cloud Based Logging

This is a guest post written by Jason Ruane, the Technical Director atCirkit.io [http://cirkit.io/], and originally appeared onthe Cirkit.io blog [http://cirkit.io/index.php/node-red-with-logentries/]. Node-Red is an app, which runs on top of NodeJs. Node-Red is usually run on cloud servers, but as more powerful CPUs are placed in IoT devices, it is only a matter of time before it can be supported locally. It already runs on Raspberry Pi and Intel Galileo. Node-Red enables programmers to build

4 min IT Ops

How to Keep CALMS and Release More!

DevOps is fundamentally changing. It is no longer considered a fad by those working with the latest development tools. Now, DevOps emphasizes people and processes as much as the cool tools, and there are many different approaches to bringing them together. If you Google search for “DevOps flow,” you will find lots of flow diagrams and methodologies. While each one highlights different priorities and names, they all have the same general themes. 1. Start with people 2. Bring in automation 3

5 min IT Ops

Which CSS Preprocessor Should You Choose?

With the growing number of CSS Preprocessors likeLESS [http://lesscss.org], Sass [http://sass-lang.com/], Turbine [http://turbine.peterkroener.de/index.php], Stylus [http://learnboost.github.io/stylus/] and Swith CSS [http://sourceforge.net/projects/switchcss/] to name a few, it’s hard to decide which to choose for your project, and which will provide the best CSS authoring. Rather than focus on evaluating every CSS Preprocessor (that would simply take too long), we will focus on the two most

3 min IT Ops

Shared Services: The Unicorn Every DevOps is Looking For

In my hunt for the mysterious DevOps practice, I’ve been let down. DevOps are hard to find. When you find them, they do not exactly do what you think they should do. Some DevOps teams only execute on automation for dev; while others are operations folks with a new name; and still others are internal consultants helping operations and developers (but not actually doing the work). In my DevOps scavenger hunt I have identified a new type of creature; shared services (aka the unicorn) What I have

4 min IT Ops

4 Alerts You Need In Case System Events Stop Occurring

“Silence is golden” This is not always true, especially when something you were expecting to happen* doesn’t* happen. However, this is true when you have a system or a service you are trying to maintain and things stop happening or go quiet. We recently developed a new service — Inactivity Alerting [https://logentries.com/product/inactivity-alerting/] — to help you with this common challenge and, as you might expect, it fires alerts when there is noted inactivity around a specific log or eve

3 min IT Ops

Why Great Logging is Key to Continuous Delivery

Guest blog post by Florian Motlik, Cofounder & CTO of Codeship Inc. Over the last years Continuous Delivery has gained a massive following with many development teams embracing the style. Companies have chosen (as with many other modern developer tools), to either build their own or embrace a hosted service like Codeship [https://codeship.com/?utm_source=Logentries]. In the end though, no matter if you go with a hosted service or roll it on your own, the goal is to move faster and build a prod

6 min IT Ops

6 Challenges Facing DevOps and Operations Teams in 2015

To say that Operations and DevOps will havea new set of challenges [/2014/05/the-challenges-of-getting-log-data-from-cloud-services/] in the coming year is a bit redundant, as their entire job is based on solving problems and facing challenges. However the current and future landscape of tools, technologies, and processes is changing dramatically. While this has always been true, it’s the pace that is problematic. Additionally, the pressure from business users who believe everything is solved w

4 min IT Ops

How to Format Nginx and Django Logs Using KVPs

Interested in learning a couple insider tips to gain more value from your logs using additional structure? Specifically, we will show how to easily introduce key-value pair (KVP) format on Nginx and Django logs. Of course, the approach is not restricted to those two types of logs, and can be applied to numerous applications. Previously, our Head of Product, Marc Concannon explained how logging information in JSON format [/2014/09/3-common-problems-solved-with-json/]makes it possible to leverag

3 min IT Ops

Logs: The Most Fine-grained Data Source

As co-founder of Logentries I am often asked – “Why Logs?” And I have to admit, upon first impression, ‘log management [https://logentries.com/doc/log-management/] and analytics’ does not seem like the sexiest space 🙂 However at Logentries [http://logentries.com]we are here to redefine that space, to provide a solution to access, manage and understand your log data that is easy to use, cost effective and intelligent (i.e. it does the hard work so you don’t have to). But that being said it sti

2 min IT Ops

Become a Windows Agent Power User from the Command Line

Last week we discussed new ways a User can install our Windows Agent from the command line [/2014/10/how-to-master-the-installation-of-the-windows-agent/]. Now we’ll explore even more goodies for Users who use the command line on Windows. Over the past month we have been updating our Windows Agent to make it more powerful for our growing customers needs. One particular area we felt needed updating was the Command Line capabilities of the Windows Agent. You can now become a Windows Agent Power

4 min IT Ops

Our Favorite Continuous Delivery Tools

We’re working hard in the Logentries towers to integrate our continuous delivery tools, so we can identify and track issues as easily as possible. This saves us time that we can spend on important things like adding new features (or playing pool!). We use a lot of continuous delivery tools to manage our development cycle, and we’ve made them interact too. We use JIRA (by Atlassian [https://www.atlassian.com/software/jira]) to plan our work, Gitlab [https://about.gitlab.com/] to manage our c

2 min IT Ops

How to Master the Installation of the Windows Agent

Over the past month we have been updating the LogentriesWindows Agent [https://logentries.com/doc/windows-agent/] to make it more powerful for our growing customers needs. One particular area we felt needed updating was the Command Line capabilities of the Windows Agent [https://logentries.com/doc/windows-agent/]. Logentries is now proud to announce that you now have even more ways to install our Windows Agent to your environment. [/content/images/le-img/2014/10/Windows-Agent-Mastering-the-In

1 min IT Ops

Connecting Logs and Metrics: Logentries and Hosted Graphite

As one of the founders of Logentries, I have had the chance to meet 1000’s of our users over the past few years . Sometimes at tech talks, meet ups, trade shows or wherever our paths have crossed :). One consistent murmur we hear is ‘graphite, graphite, graphite…’ For anyone who doesn’t know,Graphite [http://graphite.wikidot.com/] is a widely used, near real time metrics-gathering system. Although typically used to capture information like CPU or memory usage, Graphite is completely agnostic a

5 min IT Ops

In the Log Management World: Are you a Fox or Hedgehog?

I’ve recently been reading Nate Silver’s book, “The Signal and the Noise.” In the book, Silver looks at a number of areas where predictions have been made and considers how successful they have been, as well as the reasons why they have been accurate (or not). I couldn’t help but draw the similarities how most companies use log management tools [https://logentries.com/doc/log-management/] today. Silver’s particular interests are political forecasting (see www.fivethirtyeight.com [http://www.

3 min IT Ops

Heroku Buttons Rock!

As one of Heroku’s beta users I had the chance to check out the new Heroku button gallery [https://buttons.heroku.com/] last week. I must say I was very pleasantly surprised. Over the past few years, it’s fair to say Heroku have nailed their PAAS offering. They are one of the few PAAS providers to truly succeed in providing developers both an easy to deploy platform, and a full ecosystem of add ons. This allows you as a developer to get on with what you do best. Yes, you can actually spend you

3 min IT Ops

Logging from PHP Web Frameworks like Laravel

So what is Laravel [http://laravel.com/]? Laravel is a relatively new MVC web framework for PHP that was released in 2012. It’s also the most popular PHP project on Github [https://github.com/laravel/laravel] at the time of this post. Like many web frameworks, Laravel boasts an easy learning-curve to get an app up and running. This was certainly the case when I used it. While this post is focusing on logging in Laravel, its applicable to other php web frameworks, like Symfony2 [http://symfony.c

5 min IT Ops

Cloud Platforms vs. On-Prem - A Guide for the Rest of Us

Working for Cloud based businesses for the greater part of a decade now, one question invariably comes up: “Why should I move my data to the cloud?” There are always a few objections that come up as well. Following-up on a previous blog post where we discussed the business benefits of cloud computing [/2014/09/7-benefits-of-cloud-based-computing-for-non-technical-techies/], this will be a discussion on the most common objections and how to have a conversation around cloud and on-premise envir

5 min IT Ops

How to Trace Transactions Across Every Layer of Your Distributed Software Stack

APM solutions give you great visibility into any code you have control over; however, today’s systems are largely a combination of code you write along with off-the-shelf components, sitting on top of VMs/containers, and cloud-based services. Thus, full system-wide visibility requires an ability to look into your APM tool as well as log data produced from the components that you may not be able to instrument. This post offers an outline of how APM solutions work and how you can combine them wit

2 min IT Ops

E2N Reduces Anxiety and Increases Customer Satisfaction with Log Data

E2NGastro [http://gastro.e2n.de/] is a small startup in Germany, who provide a SaaS platform for restaurant management, staff management, timekeeping, and work schedules (among other services). We spoke with their CEO/CTO, Bjorn Raupach, who shared the history behind their use of log data. [/content/images/le-img/2014/09/E2N-product.jpg] Before using a log management tool [https://logentries.com/doc/log-management/] , E2N had log files, but they were difficult to locate. To access them, you had

4 min IT Ops

3 Common Problems Solved With JSON

One of the most common questions we get asked by customers is: “What’s the best way to log my data?” My answer is always: “log using JSON format wherever possible.” The next question we often get asked – (but not as much)… Yeah JSON, …Hmmm, what is JSON again? So JSON is:JavaScriptObjectNotation. But put simply, this is a way that data can be stored in a structured format, where each piece of data will usually have an identifier (known as a key) and a value (which can be in multiple forma

4 min IT Ops

7 Benefits of Cloud Computing for Non-Technical Techies

Much has been written about the cloud. What it is [http://en.wikipedia.org/wiki/Cloud_computing], where it is [http://journals.uic.edu/ojs/index.php/fm/article/view/2456/2171], how it works [http://computer.howstuffworks.com/cloud-computing/cloud-computing.htm], etc. This isn’t going to be a rehash of those questions. This is a cloud computing guide for the non-developer or non-operations guy. The goal of this post is to lay the groundwork for a further discussion: of cloud vs. on-premise, an

2 min IT Ops

Log Analysis Takes Automated Orchestration Further

Operations teams know the value of scripted infrastructure deployments. And while they can actively create, maintain, and run individual scripts, they do not have oversight on what has been run, and when. By adding log analysis, automated orchestration can go so much further, resulting in a more reliable infrastructure, with better issue response. Chef, Puppet, Salt, Ansible, Vagrant, and more; once you get to know these tools, it’s easy to crank out scripts. Once you have the original scripts

5 min IT Ops

How to Avoid the Big Data Black Hole

It takes a lot of will power, in our data obsessed world to say “too much!” However, there are many ways where too much information is destroying productivity, and actually causing bad decision making, not good. But it is hard to avoid the world of opportunities that has been opened in data collection and analysis. So how do you balance the two? The first step is to understand there is a big difference between data collection, and it’s utilization. While it seems subtle, the difference is key, a

3 min IT Ops

Scaling with RESTful Microservice Architecture

As described in a previous post [/2014/06/smart-integration-testing/] on this blog, we have been using the Dropwizard [https://dropwizard.github.io/dropwizard/] framework to quickly develop high quality, easily testable [/2014/07/smart-continuous-delivery-using-automated-testing/], RESTful microservices to expand the functionality of our product. These complement the existing multi-instance services running in our cluster and contribute to the continued scaling of the Logentries service. Both i

3 min IT Ops

3 Ways to Stop the Log Paparazzi

Every scenario is the same. A forward-thinking Operations team gets their hands on a swift log management and analysis [https://www.logentries.com] platform. Two or three individuals on the team become analytical masters. And finally Operations becomes a knowledge mecca, until the business users find out. As soon as business users know what sort of data the Operations team has at their fingertips, they can’t help but ask for some of that knowledge for presentations, reporting, user or system re

2 min IT Ops

Logging with Diamond

> “Diamonds never lie to me, For when love’s gone, They’ll luster on.” -Shirley Bassey If you haven’t heard, Diamond is a fantastic Python daemon that lets you collect system metrics. With Diamond, you can collect metrics like CPU, memory, network, i/o, load and disk usage, all in readable form. Even better, is that all this data can be sent to Logentries [http://logentries.com] where you can take advantage of our powerful search [https://logentries.com/doc/search/], alerting [https://loge

4 min IT Ops

How Log Data Could Help Win Your Fantasy Football League

Log data is a powerful resource being overlooked by businesses. The value that may be hidden [/2014/07/using-log-analysis-to-find-the-18/] in the data is so large, it is borderline reckless not to manage your log data. The main reason this is occurring, is because the beneficiaries of this information are often the end business users, who couldn’t tell you the difference between a log and a blog. So there is no urgency to invest in a solution. This is where I step in, if you are in DevOps, soft

4 min IT Ops

The Pros and Cons of Open Source Logging

I recently attended DevOpsDays Boston [http://devopsdays.org/events/2014-boston/], which is a great way to get a pulse on the latest issues and technologies in the DevOps community. DevOpsDays is split between organized talks, open sessions and hanging around chatting with the other attendees. For me, the latter is always the most valuable. Over the 2 days, I probably spoke to more than 100 different people. I’m always interested in investigating how people are logging, what they are logging,

4 min IT Ops

The ABC's of REST

[/content/images/le-img/2014/09/abcs_of_rest.png] In spite of its title, this blog is not about how to enter a relaxed or restful state, but instead, presents some of the concepts behind REST (REpresentational State Transfer) and links to more information. It will show that simply returning JSON over HTTP is not a REST API. At Logentires, we are using the RESTful style to extend the flexibility of our architecture which will allow us to make features available to users through REST APIs in the

3 min IT Ops

Happy Bits Shortens Time-to-Resolution From 4 Days to 5 Minutes!

Happy Bits [http://www.happybits.co/] develops fun mobile apps so you can share videos instantly with your friends through messenger. You can either watch the videos live while you’re recording, or later whenever you have time. Here’s how it works: you upload a video to their app, then receive a private link to paste into your favorite messaging app to share the video. Happy Bits integrates with all the popular messenger apps, like Facebook, WhatsApp, Skype, Twitter, and Google+, just to name a

3 min IT Ops

Solving the Business App Adoption Riddle

Business applications are a bit of a paradox. They are critical to business operations; you know it, and chances are all the potential business users know it as well. However getting adoption of these platforms is often like pulling teeth. Why should it be so hard to get users to leverage the platform they know they need? To make the problem worse, it might take years for an organization to know it is even a problem. Which means the investment in licensing, support, and setup for the applicati

3 min IT Ops

How to Find and Solve Your Application Bottlenecks

Refael Botbol is the head of DevOps at BlazeMeter [http://blazemeter.com/] , which does performance testing and monitoring for mobile and web apps. [/content/images/le-img/2014/08/blazemeter_logo.png] Here’s a familiar scene: You’ve finished developing your application or a neat new feature and you’ve created the right system logs for new capabilities. You’re confident that if something breaks, you can easily spot the errors and overcome them efficiently. However, you’re still leaving the inev

4 min IT Ops

Monitoring & Analyzing AWS CloudTrail Data From Multiple AWS Regions

We recently released AWS CloudTrail integration with Logentries [https://logentries.com/doc/aws-cloudtrail/] – and not surprisingly we’ve seen a significant uptick in adoption as one of our most popular integrations. My job as director of customer success is to make things as simple for our customers as possible. One question that consistently pops up, is how to collect AWS CloudTrail logs from multiple AWS regions. We follow Amazon’s best practices [http://docs.aws.amazon.com/awscloudtrail/lat

3 min IT Ops

How to Integrate Go / GoLang with Logentries

Would you like to send logs from your Go program code into your Logentries account?Thanks to the help of Gal Ben-Haim’s, bsphere Golang library for log entries [https://github.com/bsphere/le_go], Go coders can be sending their logs to Logentries in no time at all. Benefits of using this Go Lang library and implementing with your Logentries account include: * Remote viewing and analysis of your Go program log events * All your logs are sent to one location, and viewed through an easy to use

3 min IT Ops

Traditional Log Management Is Dead. Long Live Autonomic Analytics!

In this new era of cloud computing, faster and cheaper are not enough. The modern Ops “toolkit” needs a log analytics service built for cloud-based environments that offers easy log data centralization, autonomic analysis, and a real-time monitoring service for connecting distributed systems and teams. The rapid emergence and dominance of cloud-based systems have contributed to an explosive growth of machine generated log data. We have heard from our community of more than 25,000 IT and Dev O

3 min IT Ops

Real-time Alerting on Anomaly and Inactivity Made Simple.

“a·nom·a·ly” 1. Deviation or departure from the normal or common order, form, or rule. When someone is looking to be alerted when something unexpected happens within their environment they are usually referring to anomaly detection. But the problem is that it’s hard to turn a complex problem (i.e. looking for something when you are not sure what it is) into an easy to use solution. The thing about anomaly detection is that you don’t know how the issue will present itself so predicting the patte

4 min IT Ops

Let the Logs do the Talking

You are on top of your game. You have a log analysis tool [http://logentries.com] churning logs from all your applications and infrastructure. And now that you have data (LOTS and lots of data…), you are able to understand your infrastructure better than you ever did before. You might even build a dashboard or two that tells you what is going on with your IOPS and utilization, at a glance. But when it comes time to explain system status or configurations to everyone else in the organization you

6 min IT Ops

An APM Solution Divided Cannot Stand

This post originally appeared onthe Smart Bear blog [http://blog.smartbear.com/apm/an-apm-solution-divided-cannot-stand/]. To read more content like this, subscribe to the Software Quality Matters Blog [http://blog.smartbear.com/subscribe/]. [/content/images/le-img/2014/08/seperated-toolbox-600x415.jpg] Frustrations with lack of tool unification might just lead to revolution in the APM space… Application Performance Management (APM) is a broad concept, and many technologies fall under its umb

3 min IT Ops

Evolve, Don't Revolve

[/content/images/le-img/2014/08/e_volve.jpg] Logs have been around for a while, not quite as long as the wheel, but not far off. Here at Logentries, we have the mantra of evolve don’t revolve (as in don’t sit around spinning your wheels getting nowhere). We are taking this concept and looking to evolve the way you work with and think about your log data. Gone are the old days, where you only used logs to find exceptions. A new day has dawned, and the future is here, the future is Logentries (

3 min IT Ops

Why we love log management (and you should too)

It doesn’t matter if your title puts you in Ops, Dev, or DevOps, being able to track down the big insights from your data is the secret-sauce every engineer is looking for. Collecting the data is a step in the right direction, but organizing, tagging, monitoring, and reporting from the data provides the insights necessary to make business decisions. Log data is a huge piece of that puzzle, and a good log management tool can really help your business run more efficiently. Our team loves log mana

4 min IT Ops

JSON logging in Apache and Nginx with Logentries

I’m often asked on calls with our customers what is the preferred format for log data being sent to Logentries [https://logentries.com/centralize-log-data-automatically/?le_trial=json_logging_in_apache_and_nginx-logentries_blog-post_cta-create_trial&utm_campaign=json_logging_in_apache_and_nginx&utm_source=logentries_blog&utm_medium=post_cta&utm_content=create_trial] . While we pride ourselves on being the Log Management [https://logentries.com/doc/log-management/] tool that is easiest to setup a

4 min IT Ops

How D3 can help you build effective data visualizations

Data Visualization is the method of consolidating data into one collective, illustrative graphic. Traditionally, data visualization has been used for quantitative work, but ways to represent qualitative work have been shown to be equally powerful. The main goal of data visualization, is to communicate information clearly and effectively through graphical means. That doesn’t mean that it needs to look boring to be functional or extremely sophisticated to look beautiful. In order to achieve a bala

3 min IT Ops

The Log - So Important It Deserves a Book

I was very interested to see that Manning Publications have just announced the Early Access release of a new book called Unified Log Management [http://www.manning.com/dean/]. I looked through the table of contents and it was what I expected – a roll your own log management [https://logentries.com/doc/log-management/] cookbook with a list of technologies that might help (Kinesis, Redshift, Kafka etc.). Presumably it will also include Google DataFlow by the time the book ships. So it’s only reall

3 min IT Ops

Set Up Logging on Google Compute Engine in Less Than 5 Minutes

This blog was written by Logentries intern Russell Johnson, a student at MIT studying mechanical engineering and business. As developers continue to rely more and more on cloud computing services like Amazon Web Services, Google Compute Engine, and Microsoft Azure, the need for comprehensive log data management for these platforms has grown as well This post gives a quick overview of setting up Logentries [http://en.wikipedia.org/wiki/Logentries]on Google Compute Engine. Although Google offer

2 min IT Ops

Tailing All Your Logs, All in One Place

[/content/images/le-img/2014/08/markTwain.jpg] Logentries [http://en.wikipedia.org/wiki/Logentries]new Aggregated Live Tail [https://logentries.com/product/live-tail/]provides effortless, real-time visibility that you cannot get anywhere else…and is a joy to use (and I cant say the same about carrying a cat by its tail – for so many reasons). For those of you not familiar with our existing Live Tail view, Aggregated Live Tail is a simple, elegant and powerful product enhancement that enables y

6 min IT Ops

5 Ways to Make Your DevOps Team More Efficient

As your DevOps team grows, scaling efficiencies across the group is imperative to maintaining a well-oiled unit. A small team of smart engineers can function well without much effort, but as your team gets bigger, you need to make sure you invest in the right tools and practices to help keep everyone on the same page. Throw in distributed teams, across different time zones, and issues can only get exasperated. Below are a few key tips that you should consider as your development and operations

5 min IT Ops

Smart Continuous Delivery Using Automated Testing

This post is building on a recent post highlighting recommendations on how to simplify your unit testing by using the right set of tools (Smart Integration Testing with Dropwizard, Flyway and Retrofit [/2014/06/smart-integration-testing/]). As a company like Logentries [http://en.wikipedia.org/wiki/Logentries]rapidly grows, and the number of product features increases, an important question arises around maintaining the highest level quality and user experience. That level is usually where the

2 min IT Ops

4 Ways to Use Centralized Log Management for Improved Production Monitoring

Is your team focused on preventing outages and minimizing downtime in production? Time to resolution is one of the most important operational KPIs for Ops teams, becauseany time that your application is down, is too long for your users. Whether it’s minutes, hours, or (in a worst case-scenario) days, any time that your systems are down, your business is losing money; and more importantly, customer trust and satisfaction. This creates additional pressure on your team and hurts employee morale. By

3 min IT Ops

Digging Deeper on AWS: Real-time Alerting for Windows Security Events

After many conversations with our AWS customers and the AWS CloudTrail team, we recently released our AWS CloudTrail integration [https://logentries.com/logentries-and-aws-log-data-analysis/] to automatically support the most important log events our customers wanted to be monitoring across their AWS environments. We found that some of the most common need for notifications included: * Starting, stopping, terminating, rebooting instances * Creating or deletingsecuritygroups * Creating and d

2 min IT Ops

Revisiting the Past with Logentries and Syslog

When I was younger I was always amazed by my grandfather. He would revel me in stories about how, when growing up, not only did he not have a car or television, but he had to walk up hill “both ways” to get to school – without shoes! And here I am today more or less saying the same thing about the late 70’s and early 80’s when client-server changeover that occurred during that time and we all had servers sitting in our closets. Setting up syslog [https://logentries.com/doc/about-syslog/] back in

2 min IT Ops

AWS CloudWatch Logs - Making Innovation Easier and Cheaper

Last week AWS made an exciting announcement at their NYC Summit, which I believe is a big step in helping to reduce the cost of log management in the cloud [https://logentries.com/doc/log-management/] for end users and vendors alike – good news all around! What was announced? Amazon announced ‘CloudWatch Logs’ which essentially allows you to send log data from your EC2 instances into CloudWatch for storage and some rudimentary analysis. How does it work? CloudWatch logs works by collecting y

2 min Logentries

Logs to Metrics to JSON to Geckoboard

The world of log management [https://logentries.com/doc/log-management/] and application performance monitoring [/2014/04/apm-is-not-enough/]are evolving and there is no longer a big distinction between the two. Up to even a year ago, many people in IT viewed logs as simply a means of identifying and tracking issues once they have occurred within their system. It was rare that people were using their logs for more than basic application event tracking and watching for exceptions. But over the

2 min IT Ops

DeltaOne Solutions: From Reactive to Proactive with Log Management

“It’s rare to find a product that truly makes your life easier.” – Bernie Thurston, Global Head ofDeltaOne Solutions [http://www.deltaonesolutions.com/deltaone/web/me.get?web.home]. DeltaOneoffers the trading and investment management community a comprehensive technology-enabled service, delivering high quality, timely, and accurate Global Index and ETF data, across all asset classes.DeltaOne Solutionsare used in the front, middle, and back office by the world’s largest Investment Banks, Asset

3 min IT Ops

TCP or UDP for Logging – Which One to Use?

When sending data over the Internet, we want to be sure that data has arrived safely and is delivered in a reliable manner. Assuming that our hardware and operating systems do the job well, we can use transport protocol on the application level to achieve that goal. In theOSI [http://en.wikipedia.org/wiki/OSI_model] model the two most populartransport protocols [http://en.wikipedia.org/wiki/Transport_layer] areTCP [http://en.wikipedia.org/wiki/Transmission_Control_Protocol] andUDP [http://en.wik

2 min IT Ops

Using Log Analysis to Find the .18%

Even the most non-technical audiences realize the business value in analyzing their log data. DevOps professionals are constantly being asked to monitor their application performance, and often rely on log data for troubleshooting, diagnostics and application systems monitoring. The challenge is that the amount of log data generated can be completely overwhelming. Why log analysis is so difficult? In one word — volume. Even relatively small applications, can generate millions, or billions of lo

4 min Logentries

How to Get Full Visibility of CloudTrail, CloudWatch & System Logs

At re:Invent last year [/2013/11/reinvent-keynote-overview-amazon-cloudtrail-appstream-and-workspaces-announced/] AWS launched CloudTrail [http://aws.amazon.com/cloudtrail/], which is essentially a “trail” of all activity in your AWS Cloud environment… the clue is in the name ;). CloudTrail has been widely accepted as a great addition to the AWS toolset as it is useful for numerous use cases including security analysis, troubleshooting, tracking of AWS resource usage, and more. While CloudTrai

3 min Logentries

Using Log Data to Support a Mobile Business

As mobile device usage surges, and they become a top tool of choice, businesses need to be able to provide customers with easy mobile access and give employees tools to do their job while on the go. Still, questions remain: When is it necessary for a business to deploy mobile friendly interfaces or dedicated applications? And, once you’ve made the decision to do so, how do you monitor those mobile applications to make sure they are actually meeting your business and your users’ needs? [/content/

1 min IT Ops

Simple Logging to Logentries from a Docker container

Guest blog post on quick tips for logging from a Docker container by Logentries customer Gal Ben-Haim, Head of Software at Augury, @gbenhaim While building Docker images for a production deployment I encountered a problem with proper logging of what’s going on inside the containers, especially if using supervisord to run multiple processes. One simple solution can be to share the /var/log/supervisor directory with the host. Since I’m already using Logentries [http://logentries.com] for cent

0 min IT Ops

VelocityConf 2014 – Ghost Busting & Good Times

[We had an awesome few days at Velocity and loved meeting everyone who stopped by the booth, or came to the Ignite reception. Check out some of the pictures and presentations from the show.](//storify.com/trevparsons/velocityconf-2014)

4 min IT Ops

Google Cloud DataFlow – A Game Changer?

So the biggest revolution in database and analytics technology – namely the distributed batch processing technique known as MapReduce (and the associated Hadoop-centric ecosystem that has built up around it) is a legacy technology [http://venturebeat.com/2014/06/25/google-cloud-dataflow/] for one Silicon Valley player. Last week Google announced the arrival of Google Cloud Dataflow – a new service for cloud-based big data analytics that, Google says, supersedes MapReduce.[ While various VCs and

3 min IT Ops

Smart Integration Testing with Dropwizard, Flyway and Retrofit

It’s widely understood that increasing the scope and complexity of a piece of software almost always dramatically increases the effort required to verify it. Verification typically entails testing the behavior of the new feature, plus ensuring no existing functionality has been adversely affected. Because this kind of testing can quickly become painful for even the simplest of components, and because at Logentries [https://logentries.com/centralize-log-data-automatically/?le_trial=dropwizard_f

4 min IT Ops

Connecting Systems, Teams & Our Community of 25,000 Users

Ever since we processed our first log event, our vision at Logentries has always been to make log data accessible to the masses through an easy to use and intuitive technology. Why? Well mainly because we saw that log data was being totally underutilized – it’s produced by almost every piece of software from your mobile apps to large enterprise systems – and it contains so much useful data for a whole range of use cases. It was no longer simply a developer’s debug tool of last resort. Instead

3 min IT Ops

Log Annotations - It's Nice to Share

“Share your knowledge. It’s a way to achieve immortality.” – Dalai Lama I’m not saying the Dalai Lama was directly referring to our new annotations feature, but I’m pretty sure that if he needed a Log Management [https://logentries.com/doc/log-management/]tool ours would tick a lot of his boxes (mind you being Head of Products I’m probably a bit biased). Information is one of the most important assets any company has and the ability to easily share information with the right people at the righ

3 min IT Ops

Why You Should Analyze Your Logs When Load Testing

We recently hosted awebinar about running load tests [/2014/06/how-to-run-jmeter-tests-for-500k-users-analyze-log-data-results/] (with BlazeMeter [http://www.blazemeter.com]) and then analyzing the log data generated on the application under test (with Logentries [http://logentries.com] ). We had a great response and I wanted to share a few more thoughts about why load and stress testing your applications is an important step in understanding how your systems will behave in production when incr

1 min IT Ops

How to Run JMeter Tests for 500k Users & Analyze Log Data Results

Logentries recently co-hosted a webinar with BlazeMeter [http://blazemeter.com] on advanced JMeter testing for 500,000+ users. The webinar had almost 1,000 registrants, and focused on helping users understand what’s happening on their systems during load testing; identify any warnings or exceptions being thrown; and how to use log data to analyze performance under this load level. In this webinar, Logentries co-founder Trevor Parsons and BlazeMeter DevOps Team Leader Refael Botbol covered how

3 min IT Ops

Correlating Logs and Server Metrics...Finally!

Server resource usage information is a key data point that is critical for any Operations/DevOps team. I like to think of server resource usage information as analogous to the dashboard of your car — it can tell you if you are running out of gas or if something is overheating your system. Similarly, if you keep a record of your server metrics you can also use them to investigate and debug issues in your systems, understand performance trends as well as use in capacity planning over time. [/con

4 min IT Ops

Debugging Rails with Logs

This article was written by Jack Kinsella, a Logentries customer based in Berlin. Jack owns Oxbridge Notes and co-founded Bolivian Express. Below Jack has shared an excerpt from his writings on A Comprehensive Guide to Debugging Rails. Seasoned Rails pros will have interpreting logs down second nature, but I’ll outline the important realizations I had as a beginner for the sake of the version of me out there who is new to Rails development. Whenever you visit a page/fill a form/do an Ajax reque

2 min IT Ops

Collecting and Analyzing Logs from Your Java Application

le_java is one of several libraries that allow you to send log data directly to your Logentries account from your application. le_java supports both the log4j and logback logging libraries in java, and can be configured to work with either – in just a few steps! (Check out: github.com/logentries/le_java/blob/master/README.md [https://github.com/logentries/le_java/blob/master/README.md] for our complete tutorial.) [/content/images/le-img/2014/06/java.jpg] Once you have your application send

2 min IT Ops

Hitting the Jackpot with Logentries; Time to Resolution Improves by 300%

I recently had a chance to sit down with Joe Leavitt, Director of Infrastructure and Technology Products at Diamond Game [http://www.diamondgame.com] to chat about how he is using Logentries today to monitor and manage his business. Diamond Game is a Class II gaming, Class III gaming and LT-3 ITVM manufacturer that also designs and services the games and the machines that they run on. Their machines and games are often found in commercial casinos, racetracks, and bingo markets. [/content/ima

3 min IT Ops

Managing DevOps Across Multiple Clouds

Like it or not the future of enterprise IT involves hybrid cloud computing in multiple forms. On the one hand, IT organizations will be trying to cope with deploying applications on private clouds that might be running in their own data center or in a third-party hosting facility. At the same time, a large number of applications are just as likely to be running on multiple public clouds. Once deployed, those applications are not likely to move very often; but from a DevOps perspective the fact

1 min IT Ops

Velocity Conference: Highlights from 2013 & What to Expect this Year

#VelocityConf: Highlights from 2013 & What to Expect this Year We are excited to be heading to #VelocityConf 2014 in Santa Clara from June 24-26th. Here are some of the highlights from last year's #VelocityConf. What are you looking forward to this year? 2013 Velocity Video Highlights: Did you miss Velocity Conference last year? Watch the keynotes, lightening demos, interviews and more below. Some highlights of Velocity 2013 including Kate Matsudaira's speech "If You Don't Understand People,

4 min IT Ops

How to Get Useful Log Data from Your Cloud Services

So you’re generating lots of log data from your cloud services like Google, Amazon and Microsoft. And you know there is a lot of value in using that data to optimize and troubleshoot those services. But, how exactly do you get access to that useful information and immediately apply it to your business operations? In our earlier post, “The Challenges of Getting Log Data from Cloud Services [/2014/05/the-challenges-of-getting-log-data-from-cloud-services/],” we covered the different ways three le

2 min IT Ops

Logentries New Integration for Windows Phone

One of our most popular libraries at Logentries is our .NET integration. We offer support for both Nlog and Log4net. We want to continue to expand Logentries support for .NET environments, and we are pleased to announce our new integration for Windows Phone. With the Windows Phone logging module you can now easily centralize, search and analyze important log events that occur in your app. Setting up the Log Appender in your code is easy. Simply inject this initialization method when your app la

2 min IT Ops

Monitoring the Most Common Tags and Alerts in AWS CloudTrail

AWS CloudTrail is a web service that records AWS API calls for your AWS account and places these records in log files stored in an S3 bucket of your choice. The recorded information includes the identity of the API caller, the time of the API call, the source IP address of the API caller, the request parameters, and the response elements returned by the AWS service. With CloudTrail, you can get a history of AWS API calls for your account, including API calls made via the AWS Management Console,

3 min IT Ops

Velocity 2014 - What We're Looking Forward To

A few highlights that we are looking forward - great speakers, good discussions and some fun events. We'll keep track of them here, so keep checking back in. And, we want to hear from you so jump in and let us know what you're looking forward to! I'm excited to announce that the Logentries [logentries.com] team will be attending #VelocityConf [https://twitter.com/search?q=%23VelocityConf] 2014 in Santa Clara. We are looking forward to sharing ideas about building faster, stronger web apps, and

3 min IT Ops

The Challenges of Getting Log Data from Cloud Services

It’s not surprising that the active use of cloud services can generate an immense amount of log data, or that analyzing that data can allow you to more effectively deploy and utilize your cloud services. But what might surprise you is the different ways the major cloud service providers generate log data and how they suggest you utilize it. [/content/images/le-img/2014/05/log-data-from-the-cloud.jpeg] As an example, Google is probably the best known of the cloud service providers, withthe Goo

2 min IT Ops

Configuration-free Auto Logging

You can now send and process logs without ANY configuration! And thus, signing up and playing with your logs has never been easier. Now 100% of our users that sign-up automatically send data, within seconds, and get to see real log data immediately without having to configure a single server. ‘How??’ I hear you ask…? Well, one of our engineers came up with a clever idea, that we like to call auto-logging [https://logentries.com/real-user-monitoring/]. It takes advantage of our javascript libra

2 min IT Ops

Our Favorite Linux Performance Monitoring Tools

As a part of monitoring and troubleshooting our system and applications, we often need to get a quick snapshot of information about the status of our server. This usually means SSHing into a server and running certain commands to get to the information we need. I wanted to share a quick overview of my top 5 commands to get a snapshot of this server information. htop [http://hisham.hm/htop/] htop is an ncurses-based interactive process viewer for Linux. It gives information about the CPU utiliz

1 min IT Ops

Making Azure, ASP.NET, and MVC Logging Look Easy

Coming straight to you from our customer success team is a great story from one of our customers, Max Vasilyev, who is a Senior Developer at an oil servicing company in Aberdeen, Scotland. A few months ago, Max deployed a multi-tenant HR application on Microsoft Azure [http://en.wikipedia.org/wiki/Microsoft_Azure] developed in ASP.NET, MVC, and Autofac. [/content/images/le-img/2014/05/azure.jpg] With more than 180,000 lines of code, this application uses token based logging [https://logentri

2 min IT Ops

Trends in Mobile Log Data Usage

Managing mobile devices can be quite an adventure. With the rampant penetration of smartphones into the business market, it has become necessary for IT to add yet another device to the long list of potential headaches in terms of support and security risks. Add BYOD into that mix and it gets even more complex. Making use of log data from smartphones and their applications just makes sense, especially when these devices are used to remotely access business resources. Some of the most important t

6 min IT Ops

Logging - What's Your App Doing Right Now?

Geoff is a Logentries user and the Systems Development Team Leader for the GeoNet Project [http://geonet.org.nz/]atGNS Science [http://www.gns.cri.nz/]in New Zealand. Knowing how your application is performing in production should be really important to you. I’ve written before aboutgathering application metrics [http://geonet-dev.blogspot.co.nz/2014/03/metrics-whats-your-app-doing-right-now.html] . This post is about gathering log messages. I’ve started writing this post twice before and it

2 min IT Ops

Using D3.js to Graph Your Log Data

At Logentries, we use the open-source D3.js visualization library for a number of our graphs, including our recently released Insights feature [/2014/05/create-dashboards-from-your-logs-data/]. In a nutshell, D3 allows you to efficiently manipulate documents based on data with minimal overhead. While it could in fact be used for all types of DOM manipulation that you might do with jQuery instead for example, we have used it purely for its graphing functionality. One of the reasons we chose D3 i

3 min IT Ops

Customer Care Is Not Optional

Here at Logentries, customer care is not optional. We support all of our customers, whether they’re paying subscribers or users of our free service. The Logentries team is extremely passionate about the power of log data and the impact the insights derived from log data can have on development, IT and business operations. As part of our mission, we’re working hard to make log data and these insights accessible to virtually anyone. Over the past several months we’ve released a number of new capa

3 min IT Ops

Log Management and Analytics: Strength in Numbers

The number in this case is a community of more than 25,000 users of log management [https://logentries.com/doc/log-management/] and analytics. A collection of smart individuals and businesses from the small to the very, very large who have all turned to Logentries to centralize and analyze their logs data. This is not just a random group of people, but a very technical community with a common core requirement: the need to process their log data, extract critical information and be alerted in

2 min IT Ops

3 Smart Ways Businesses are Using Log Data

Log data is everywhere. From mobile apps and desktops, to servers and massively distributed applications, just about everything generates log data that can be useful for optimizing IT processes. But with the focus on IT, we can sometimes overlook the potential for finding business improvements by analyzing that same data we usually use for IT problem solving. 1. Improve the Customer Experience One of the simplest and most powerful uses for log data is to analyze user behavior information from

3 min IT Ops

Musings on the Future of Docker

We’ve had a few posts recently on Docker (see here [/2014/03/the-state-of-logging-on-docker/], here [/2014/03/how-to-run-rsyslog-in-a-docker-container-for-logging/]) and CoreOS (and here [/2014/04/sending-your-coreos-log-data/]) and generally we’ve focused on the details of how you can easily get your Docker logs sent up to Logentries. But recently I’ve been thinking beyond just how to log with Docker, and more about Docker Inc. and what comes next. So as we all watch and wait to see what happe

1 min IT Ops

Strong Encryption on Logentries Endpoints

Recent disclosure of the highly publicized Heartbleed [https://en.wikipedia.org/wiki/Heartbleed] bug again brings attention to security of communication over the Internet. Security of the Logentries service and the customer data we store has always been paramount for us, and we believe that security is a shared responsibility. To that point, I want to share with you the steps we take toward this contract. First of all, we don’t support weak encryption. This means that if you decide to send us

5 min IT Ops

Why Creating Real-time Dashboards From Your Log Data is a No-brainer!

I recently came across a blog post on OpenOpsIQ [http://openopsiq.com/2014/04/03/apm-logging-monitoring-three-legs-of-a-stool-or-redundant-tools-waiting-to-be-consolidated/] asking the question: “With the introduction of real-time logging, why cant we have a single solution for monitoring the entire stack?” In my opinion, you can! As we have seen lately from a survey we carried out across a sample of the Logentries’ 25,000+ user base [http://www.bizjournals.com/boston/blog/startups/2014/04/log

1 min IT Ops

Customer Success: Using Log Data to Respond to Security Incidents

One of the more unique customer use cases we’ve seen recently is allowing the security incident response team to use log data to track malicious attempts against their network. It is well established that Logentries can maintain log data across all of your systems – but adding just a few more pieces of information can result in deep insights into your security measures and their effectiveness. So how do you do this today? Most current generation firewalls and routers support syslog capabilitie

2 min IT Ops

Log Aggregation & Grouping in 3 Clicks

With the introduction of a centralized,aggregated view of your logs [https://logentries.com/doc/ui/#Aggregated_View], we enabled our users to see their entire stack in one view and to quickly correlate different logs together. This log aggregation viewpoint provides much deeper insight into what is occurring across your logs from various sources. We thought that the Log Aggregation View was pretty cool, so we are very excited to launch log Groups which add even more flexibility to save and moni

1 min IT Ops

Customer Success: Sending your CoreOS data to Logentries

We love sharing cool stories straight from our customers about how they are using logs to solve problems. This one is from Matthias Kadenbach (@mkadenbach) who was using CoreOS with multiple Docker containers on Google Compute Engine and not sure how to log from CoreOS to remote destinations. CoreOS is a stripped down version of Linux (Chrome OS) that has no package manager. This basically means no build-essentials are included with the OS, which makes it impossible to download and build the L

2 min IT Ops

Log Management Best Practices: Alerting on More Than Exceptions

In ye olden times, log-based monitoring was narrowly focused on tracking down and identifying exceptions. But those days are long gone, and logs are becoming the defacto method of extracting crucial data from your systems, be this the traditional error and exception identification; monitoring the performance of servers and your application; and now evaluating your customer’s usage of your application. [/content/images/le-img/2014/04/24_7_anouncements.jpg]Today, logs can provide deep analysis b

3 min IT Ops

4 Reasons to Love Your Log Data

Data logging by essential IT equipment has been around since the beginning of the modern computer era. Operating systems, application software, hardware, and a plethora of IT equipment in and on the network generate log files, and IT professionals can often find themselves knee deep in an overwhelming amount of data, especially as cloud services are added to the mix [/2014/04/apm-is-not-enough/]. But, the truth is that if used properly, log data can be a very good friend to both IT and business

4 min IT Ops

What is "real-time" anyway…?

I love a good buzzword…cloud, big data, analytics …And even more than the buzzwords, I love the liberties people tend to take applying these buzzwords to their new systems and services. Such buzzwords regularly get abused and often get washed into marketing material and product websites in an attempt to hoodwink and woo new unsuspecting customers. One of my (least) favorite buzzwords, that I’ve noticed popping up more recently in particular in the logging space is “real-time.” So what does re

4 min IT Ops

In At The Deep End: Training For High Performance Distributed Systems

Here at Logentries we have a simple philosophy when it comes to hiring: hire the best people we can find and let them jump in at the deep end. That is how we like to learn. Smart people like to go deep and then find out what they don’t know as they work through some real world problems. And, our job is to give them the mentoring and support they need to overcome the blockers quickly and continue the learning process. We ensure they come to us with a great computer science or mathematical backgro

1 min IT Ops

A Note on Logentries Security

The recent OpenSSL vulnerability CVE-2014-0160 [http://www.openssl.org/news/secadv_20140407.txt], nicknamed “Heartbleed,” affected large part of the Internet. It was caused by a relatively trivial bug, a missing check for an input value, which can lead to a buffer overrun, causing leaking of an unrelated block of memory. This can ultimately lead to compromising of the secret keys used to encrypt the traffic, which essentially allows attackers to eavesdrop on communications, steal data directly f

3 min IT Ops

Automating logging to Logentries

Staying on the subject of devops, specifically server automation and monitoring, I’m going to show you how you can automatically send logs to Logentries using Chef [http://www.getchef.com/chef/] and Vagrant [http://www.vagrantup.com/]. If you are unfamiliar with either of these technologies I suggest you have a look through my previous posts [/author/joe/] to bring you up to speed on things. [/content/images/le-img/2014/04/logentries-chef.png] We’re going to cover how to install the logentrie

2 min IT Ops

Saving Searches and Saving Time

We encourage our customers to always share new feature ideas with us, and one common request we have been hearing is for the ability to save log searches so they can easily be run later. We’re happy to announce that this feature has just been released! I wanted to share why I think saved searches is an exciting enhancement to the Logentries service. With saved searches, you no longer need to remember or store your complex searches. You can easily recall them in one click and run them instantly

2 min IT Ops

The Value of "Easy Peasy"

One customer recently summed up our service, “With Logentries I don’t have to SSH to a server to get logs and debug a problem, it’s easy to get to the logs. No need for remote desktop or anything like that…Easy peasy!” [/content/images/le-img/2014/04/Easy-Peasy-pic.png]I love this description because it is exactly what we intended. While we may not have used those words, we are on a mission to deliver the easiest, most accessible log management [https://logentries.com/doc/log-management/] and

4 min IT Ops

An APM Solution is Not Enough

As a long time APM guy, and for someone who has spent quite a bit of time building monitoring tools such as transaction tracers, memory analyzers, tools for analyzing garbage collection behavior etc. it pains me to say this, but sometimes an APM solution is not enough. In fact, a recent blog I read byAndreas Grabner [http://apmblog.compuware.com/author/andreas-grabner/]of Compuware entitled “Don‘t Trust Your Log Files: How and Why to Monitor ALL Exceptions” got me thinking on this topic. Grabne

5 min IT Ops

Logs As Data - What Are the Top Use Cases For Your Logs?

The humble log has largely been thought of as a data source that is there “just in case” there is a problem: a record of what has happened, that provides the evidence needed, when you are required to investigate a system issue or security event, for example. Generally logs have only ever been accessible by the chosen few, who sit in the engine room of the system looking at at log events flying past all day long – think of that scene in the Keanu Reeves movie “The Matrix,” where the character Cy

5 min IT Ops

Tracking Events Through Complex Stacks – Part 2

In part 1 of this post [/2014/03/tracking-events-through-complex-stacks-part-1/] , I went through the general issue of how systems have become more complex, utilizing more and more components and frameworks from 3rd parties. The disadvantage of this being the loss of insight into what’s happening behind all these closed doors, increasing the difficulty of identifying issues. In this post, I’ll go through how you can solve this issue. By tracing transactions as they traverse your stack, you can

2 min IT Ops

5 Great Blogs for DevOps

If you are a DevOps professional looking to get more involved or further your learning, or just looking for some entertaining, insightful content, we wanted to put together a list of 5 great blogs for DevOps. And, we know we missed some, so feel free comment and share what other blogs should be on here! A few great blogs that we think you might find valuable: 1. ContinuousDelivery.com [http://continuousdelivery.com/] – Dave Farley and Jez Humble run this site. Jez is a principal consult

3 min IT Ops

HipChat, PagerDuty & Campfire Integrations - Webhooks

We’ve had a lot of users ask for third-party integrations for our Tags & Alerts [https://logentries.com/doc/tagsalerts/] lately…so we delivered! You can now configure your Logentries Alerts to be sent to Campfire, HipChat, and PagerDuty, plus there’s plenty more to come. [/content/images/le-img/2014/03/webhooks-lg.png] This allows our users to easily aggregate alerts from across their stack into a centralized forum that they can share with their team. [/content/images/le-img/2014/03/webhooks

2 min IT Ops

Tracking Events Through Complex Stacks - Part 1

The complexity of companies’ software solutions are increasing, with developers having less and less insight into all the corners of an application. Years ago, when I was initially developing (now I’m talking 20 years ago here), I remember how a lot of business-facing or customer-facing applications were built on a rather simple stack. Often, applications were built on an application server (such as Tomcat, ColdFusion or Microsoft’s ASP server) with a SQL database behind this and utilized web b

4 min IT Ops

4 Stages of Rolling Your Own Logging Solution - When will you jump off the Complexity Elevator?

Over the past few years we have brought on board many a company that has started out on the path of rolling their own logging solution. Very often companies start down this path…largely because they can (there’s a bunch of open source technologies)…and it’s free, so you can get started with zero down. But as we all know there’s no such thing as a free lunch, and rolling your own solution contains a number of hidden costs, such as some beefy servers for when your log volumes grow, your VALUABLE

1 min IT Ops

Extending Your Front-End Insights with The Logentries Pixel Tracker

Since we rolled out our JavaScript logging capability [https://logentries.com/doc/javascript/] last Fall we’ve seen the concept of client-side logging (both on the web and native platforms) gaining a lot of traction as customers try to get more visibility into how people use their products [/2013/09/how-to-log-client-side-javascript-events-logging-for-the-web-with-le-js/] , and quickly troubleshoot them when things go wrong. We use the JS client library [https://github.com/logentries/le_js] ext

2 min IT Ops

Back to the Future with Old School DevOps

Have you ever carried a pager? Felt that ominous vibration on your hip when you least expected it? Been out at dinner when you get a message the systems aren’t working, and you need to figure out why? If so, hopefully this story will resonate with you, and illustrate why I believe what we are doing at Logentries [https://logentries.com] matters. [/content/images/le-img/2014/03/Pager.jpg] In my case, I was enjoying a beer with a friend when the “chocolate bar emergency” (as it will be known

6 min IT Ops

Synchronizing Clocks In a Cassandra Cluster Pt. 2 - Solutions

This is the second part of a two part series. Before you read this, you should go back and read the original article, “Synchronizing Clocks In a Cassandra Cluster Pt. 1 – The Problem [/2014/03/synchronizing-clocks-in-a-cassandra-cluster-pt-1-the-problem/].” In it, I covered how important clocks are and how bad clocks can be in virtualized systems (like Amazon EC2) today. In today’s installment, I’m going to cover some disadvantages of off-the-shelf NTP installations, and how to overcome them. C

5 min IT Ops

Synchronizing Clocks In a Cassandra Cluster Pt. 1 - The Problem

Cassandra [https://cassandra.apache.org/] is a highly-distributable NoSQL database with tunable consistency. What makes it highly distributable makes it also, in part, vulnerable: the whole deployment must run on synchronized clocks. It’s quite surprising that, given how crucial this is, it is not covered sufficiently in literature. And, if it is, it simply refers to installation of a NTP daemon on each node which – if followed blindly – leads to really bad consequences. You will find blog post

3 min IT Ops

DevOps: Vagrant with Chef-Server

In my last blog post, The DevOps Tools We Use & How We Use Them [/2014/02/the-devops-tools-we-use-how-we-use-them/], I talked about how we use Chef [http://www.getchef.com/chef/] with Vagrant [http://www.vagrantup.com/] for managing, maintaining and monitoring our servers. (If you haven’t read it yet, I suggest you have a quick look at it for reference.) This is great for a development environment where you spin up servers locally in Virtualbox, however in doesn’t help much when trying to maint

3 min IT Ops

The State of Logging on Docker

Last week I wrote a post on how to log from Docker containers [/2014/03/how-to-run-rsyslog-in-a-docker-container-for-logging/]. In short I was playing around with Docker [http://www.docker.com/] and wanted to try to figure out how to get some log data from my containers into my Logentries account. I tried the following approaches: * Docker ‘logs’: Docker has a ‘logs’ command $ docker logs Container_IDthat will fetch the logs from a container. You can run this via the docker daemon on you

4 min IT Ops

Dogfooding Our Own Solution: Cobblers, Pet Food & Jidoka

When you are in the log management [https://logentries.com/doc/log-management/] business, who manages your logs? Well, you do, of course. The proverbs of old, though, tell you this can sometimes be a very bad idea. In the Irish language the saying is: “Ba mhinic droch-éadach ar tháilliúr ‘s droch-bhróg ar ghréasaidh. [http://en.wikiquote.org/wiki/Irish_proverbs]” Or, in English, “Often the tailor has bad clothes and the cobbler has bad shoes.” The Scots’ phrase is very similar: “The blacksmith

3 min IT Ops

Archiving Data: Are You Vulnerable?

Archiving of your important data is something a lot of people and companies overlook. Data is probably one of the most vital assets of any company and is the lifeblood that keeps your business moving. Data loss is usually not on the radar until it happens. And, once it does happen, the importance of good backup/archiving becomes brutally apparent. Are you prepared? As a simple experiment, imagine that a certain system within your organization was subject to a small fire, all the machines and

2 min IT Ops

Windows Servers: How to Get CPU, Disk, Network, and Memory Data

This is for our ever expanding group of users out there that use Windows servers to manage their applications and services. You may think of Linux (in its various flavors) as being the primary OS of the Cloud, but you’d be surprised. We’re seeing more and more applications and services being hosted on Windows platforms all the time. And the easiest way for you to send your logs to us is by using the Logentries Windows Agent [https://logentries.com/doc/agent/]. Some of you may be saying, “but yo

2 min IT Ops

How To Run Rsyslog in a Docker Container for Logging

I’ve been playing around with Docker [https://www.docker.io/] this morning (read as I have followed their 15 min tutorial and have installed it on an Ubuntu instance – so I’m not quite the expert yet). I was initially interested in figuring out what log management [https://logentries.com/doc/log-management/] looks like for any Docker users out there. From first look, Docker has a “logs” command that will fetch the logs from a container. You can run this via the docker daemon and it will captu

2 min IT Ops

Real User Monitoring: I Need Insight And I Need It Now

In the past it was assumed that the web-based interface was the most important, and often the only, path for a user to access content or a product. But those days are gone and now companies must embrace supporting multiple interfaces on different platforms in order to satisfy their users. With customers looking to use a mix of clients, with the most common being Web, iOS, Android and Windows Phones, companies need to invest in optimizing for these channels and, hence, protecting their investment

3 min IT Ops

Help for Moving From Physical Servers to the Cloud

A few weeks back, I was reading through Hacker News [https://news.ycombinator.com/news]when I came across an article titled “AWS Tips I Wish I’d Known Before I Started [http://wblinks.com/notes/aws-tips-i-wish-id-known-before-i-started/].” Expecting a “Top 5,” or even “Top 10,” list I was surprised to find an extremely nuanced, approximately 50 point list with details on each. Which would make sense, then, why I found it at the top of Hacker News that day. As I read through it, though, I was hap

6 min IT Ops

Mobile Apps: 6 Steps to A Wealth of Untapped Data

With mobile technology becoming a bigger part of many companies’ front end coverage, it makes sense to ensure that you are getting a good return on your investment by delivering a high quality app that works for your users. With this in mind we are excited to announce the release of our iOS and Android libraries [https://logentries.com/insights/mobile/]. When used in conjunction with our JavaScript library, these will give you coverage over virtually your whole front end. Mobile adoption for U

3 min IT Ops

An Inflection Point for Mobile Devices & App Monitoring

You may not have noticed, but history has unfolded before our very eyes in the past few months, with mobile device sales expected to outpace those of the traditional laptops for the first time (at least according to IDC research last September [http://www.idc.com/getdoc.jsp?containerId=prUS24314413]). To shed some light on the expected pace of change, by 2017 it’s anticipated that 87% of the worldwide smart connected device market will be tablets and smartphones, with PCs (both desktop and lapt

3 min IT Ops

The DevOps Tools We Use & How We Use Them

At Logentries [https://Logentries.com] we use an array of tools for managing, maintaining and monitoring our servers. Using current popular parlance, these could often be referred to as DevOps tools, or tools with DevOps use-cases. We’re going to have a quick look at the automation side of things, more specifically, how we spin up VMs in both development and production using Chef allowing us to have consistent software version, configurations and optimizations. We are avid users of Vagrant [htt

2 min IT Ops

What's so great about AWS S3 Archiving?

Why do so many organizations archive gigs and gigs of log data to Amazon’s S3 [http://aws.amazon.com/s3/]? Over the past few years I have spoken to hundreds (if not thousands…who’s counting anyhow … 😉 of our new users as they have on-boarded with Logentries. One of the first things that strikes me is that, even if they do not have a sophisticated log management [https://logentries.com/doc/log-management/] solution in place, so many organizations simply dump their logs to S3. As roll your own

2 min IT Ops

No Firewalls at 37,000 Feet

It never ceases to amaze me that I can be 37,000 feet over the Atlantic in a tin box (writing this blog…) traveling at a speed of 594mph, with an outside temperature of -79F, and yet I can fire log events from my MacBook Air (excuse the pun) to Logentries [https://logentries.com] and see them arrive instantaneously in my live tail view [/2013/05/tail-f-is-dead-long-live-tail-f/] . It makes me wonder what speed the log events are traveling back to our servers at and exactly how quickly they are b

2 min IT Ops

How to Monitor the Health of Your Application and Infrastructure With the Same Solution

With an attractive total cost of ownership of cloud-based solutions versus on-premise solutions, a highly flexible nature and the ability to scale up rapidly, there has been a steady increase in companies moving infrastructure out of racks and into the cloud. As more companies move their mission critical systems out of their physical control to these cloud-based solutions, it only makes sense that their application and machine generated logs are even more important than ever before. With that

2 min IT Ops

Amazon S3 Archiving...You asked, we delivered!

One thing we like to think we do well here at Logentries is listen to our global user base [/2014/01/global-tags-alerts/]! Something that a lot of you have asked for recently is log archiving. Ask and you shall receive – we have now opened a Beta for archiving to Amazon S3. So you can be safe in the knowledge that your log files will be archived safely every night to a reliable storage facility should you ever need them in the future. You can also be sure that any compliance or PCI requirements

5 min IT Ops

Our Tags and Alerts Have Gone Global

2013 was a pretty exciting year at Logentries as we made it our mission to help you find valuable insights in more and more of your log data. Over the past year we were able to grow our user base, and almost solely through word of mouth [http://www.huffingtonpost.com/elaine-thompson/5-things_b_4232386.html]. I think it’s fair to say that our service has gone global – it’s now in use across over 100 different countries. Amazingly, this is all down to our users who continue to spread the good word

4 min IT Ops

How to Send Log Data via a Proxy Server using Rsyslog

If you have a large number of servers you may have a requirement to configure proxy servers in various parts of your network to accept messages and then forward them to a centralized logging solution or service [https://logentries.com]. We regularly see this with larger customers where it makes sense to collect logs centrally before sending them on to Logentries for analysis. In some cases this can be a more stringent requirement where parts of your network may not have direct access to the publ

3 min IT Ops

How To Monitor Windows with Logentries

We are often asked at Support questions such as, “How do we use Logentries to monitor Windows performance stats?” or “Can Logentries get alerts based on CPU utilization on Windows?” The simple answer is yes, with the help of our Logentries Agent for Windows [https://logentries.com/doc/agent/] and PowerShell we can. Here is a 5 minute tutorial to get you monitoring your CPU, Memory and Disk Space [https://Logentries.com/product/why-use-us/] with Logentries. Installing the Logentries Agent You

1 min IT Ops

Integrating Logentries and Scala

On my personal blog a few months ago, I wrote about integrating Clojure with Logentries [http://chris.mowforth.com/posts/2013/logentries-clojure/]. As I mentioned there, if you write code in a language that targets the JVM, you can almost definitely send your log data to Logentries by leveraging our existing Java language support [https://github.com/logentries/le_java]. Scala has a plethora of idiomatic wrappers for classical Java logging libraries, my favorite of which would be Typesafe’s sca

3 min IT Ops

5 Tricks to Become a Logentries Power User

While we here at Logentries use the product every day, even we sometimes miss features meant to make the product more powerful and easier to use. This came to my attention the other day when one of my colleagues said to me, “you know you can highlight a pattern to create a tag, right?” My answer was a resounding, but excited, “no.” And it turns out that there’s more than just this. So, if you’ve ever wanted to use Logentries more efficiently (and who hasn’t?!) here’s a list of five tricks to bec

6 min IT Ops

How to Configure Rsyslog with Any Log File; Agents Bad...No Agents Good...

Last week I wrote “In Defense of the Agent [/2013/12/in-defense-of-the-agent/].” One of the main advantages of using agents is the ability to easily get the agent configured to monitor logs of any type [https://Logentries.com/product/why-use-us/] no matter where those logs live on your file system. We posted the article on Reddit and there were some interestingcomments [http://www.reddit.com/r/sysadmin/comments/1sjyic/in_defense_of_the_agent/] and discussion – it’s fairly obvious that there is

3 min IT Ops

5 Uses for Log Data That You Never Thought Of

When you think of logs, what do you think of? It’s most likely troubleshooting software applications and the infrastructure that underlies them, keeping an eye on your production apps…perhaps even database logs and some other things like that. Traditional log management [https://logentries.com/] stuff…I’m guessing it’s not sports cars, law enforcement, lighting, marketing metrics, and beer. Well guess what? It can be! 1) Fact Check a Journalist Back in February of 2013 The New York Times publis

3 min IT Ops

Big Data Is Getting Bigger and Faster - How We Help You Handle It

The Gartner Group [http://www.gartner.com/it-glossary/big-data/] believes that Big Data will be the new normal in the near future and defines it as: > “Big data” is high-volume, -velocity and -variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making. The 3 ‘V’’s of big data, Volume, Velocity and Variety. Here at Logentries we identified this pattern back in 2007 and since then have been focused on delivering a

5 min IT Ops

In Defense of the Agent

Last week we published an article entitled ‘Log Management 101 – Where Do Logs Come From? [/2013/12/log-management-101-where-do-logs-come-from/]‘ to which one of our more witty readers retorted: “Sometimes a server and an app love each other very very much…” 🙂 [/content/images/le-img/2013/12/Screen-Shot-2013-12-10-at-6.07.50-AM.png] The article covered some of the basics around collecting log data from various parts of the stack as shown here. [/content/images/le-img/2013/12/Application_

5 min IT Ops

Log Management 101 - Where Do Logs Come From?

We’ve had a lot of people asking for the Log Management [https://logentries.com] Primer for a while now. And, surprisingly, many of these folks have a strong technical background, including developers. Some want it for themselves, and some want it to pass on to a colleague, manager, etc. I’m going to explain what logs are, where they come from and how you can get your logs. If you’re a developer, this post probably isn’t for you as we don’t dig into the code level nitty gritty, but it will giv

2 min IT Ops

Announcing Our New Convenient & Powerful Tags and Alerts

Here at Logentries [https://logentries.com] we have been hard at work developing an improved and more powerful version of our product by making it possible for you to manage your monitoring functionality at a global level so that you can more easily assign tags and alerts across your systems. Shortly we will be releasing this new and enhanced Tags and Alerts [https://logentries.com/doc/alerts/] functionality which now gives you the ability to assign a Tag or an Alert to an individual, multiple

3 min IT Ops

Where Are My AWS Logs?

Over my time at Logentries, we’ve had users contact us about where to find their logs while they were setting up Logentries [https://logentries.com/centralize-log-data-automatically/?le_trial=aws_logs-logentries_blog-post_cta-create_trial&utm_campaign=aws_logs&utm_source=logentries_blog&utm_medium=post_cta&utm_content=create_trial] . As a result, we recently released a feature for Amazon Web Services called the AWS Connector, which automatically discovers your log files across your Linux EC2 ins

2 min IT Ops

Re:invent Keynote Day 2: Verner talks innovation and announces Kinesis

This morning’s keynote was given by Amazon CTO Verner Vogels and, similar to Andy Jassy’s keynote from day 1, it was made up of a number of new service announcements, customer stories and a look at how AWS innovate across a number of key areas. The most notable announcements from this morning were: * Amazon Kinesis: A real-tme data stream processing service * New instance types (C3 and I2) with SSD. C3 target compute intensive workloads and I2 give high I/O performance * Cross Region sna

2 min IT Ops

Announcing OpsStream for AWS - Combining Cloudwatch, CloudTrail & Trusted Advisor

What is OpsStream? OpsStream is a new service provided by Logentries [https://logentries.com/insights/aws/] that collects information about AWS services such as EC2, EBS, RDS, CloudWatch, Trusted Advisor*,CloudTrail [http://aws.amazon.com/cloudtrail/]*, etc. and routes this information into your Logentries account along with all your system and application logs. Once you signup for OpsStream [https://logentries.com/insights/aws/], data containing status, usage & user/API activity information of

5 min IT Ops

Re:invent Keynote Overview… Amazon CloudTrail, AppStream and WorkSpaces announced

I’m fresh out of re:invent’s keynote delivered by Amazon’s Andy Jassy and all I can say is WOW – what a great start to the conference. A conference which incidentally is being attended by over 9000 people and with another 9000 watching online from over 57 countries. Not surprising given AWS boasts customer numbers in the 100,000’s and according to Garnter’s latest IAAS magic quadrant has more than 5X the deployed infrastructure that the next 14 vendors combined. AWS is no longer simply the hom

4 min IT Ops

Logentries Partner Add-ons Program

Last week we released our first research report, documenting how for a particular DevOps use case 99% of log data looked like signal noise. More precisely a given set of error codes that related to performance and reliability made up less than 1% of the overall log data. That’s not to say that the other 99% of log data is useless – on the contrary, in fact, it is a constant stream of data flowing from your systems with a lot of valuable info – and it usually contains valuable data for security,

2 min IT Ops

How to Dynamically Pick Up Logs When Scaling Your Amazon Web Services EC2 Environment

I recently wrote about how to easily get all your logs from [/2013/10/how-to-easily-get-all-your-logs-from-amazon-web-services-ec2/] A [/2013/10/how-to-easily-get-all-your-logs-from-amazon-web-services-ec2/] mazon Web Services’ EC2 [/2013/10/how-to-easily-get-all-your-logs-from-amazon-web-services-ec2/]. The point with that article was to show how easy it is to instrument your EC2 application for logging and forwarding to Logentries. But the whole point of using AWS EC2 is the ability to scale q

2 min IT Ops

Digging into 22 Billion Heroku Log Entries...Forget Big Data, It's the Little Data That Matters!

At Logentries [https://logentries.com] we process over 10 billion log events every day. That’s quite a lot of data from quite a lot of systems…all being processed and analyzed by our cloud service. We realized that this puts us in a unique position to look at a huge amount of data from a macro level and provide our community with insights as to what is happening across different platforms and software components (of course, all anonymized with privacy protected). So our Logentries research team

3 min IT Ops

Log Maths: Why You Don't Want To Play "Spot the Error"

To start this post, we’re going to play a game called “Spot the Error.” Can you spot the error? [/content/images/le-img/2013/10/25_Logs_No_indicator3.png] Did you find it? How long did it take you? And this is only 25 log events! (If you couldn’t find it, click here to see the solution [/content/images/le-img/2013/10/25_Logs_With_indicator3.png].) In 2012, according to Marcia Conner [http://marciaconner.com/blog/data-on-big-data/] based on research done for her Fast Company article, “Time to

3 min IT Ops

How to Easily Get All Your Logs from AWS EC2

Let’s say that you, like many of your colleagues, are hosting your application on AWS’s EC2 cloud infrastructure. You’re chugging along at a steady rate of growth when BAM! One day you get a spike of traffic and have to scale up quickly. “Good job,” you think as you pat yourself on the back in your mind, “this choice to host in the cloud means we can easily handle this load spike without a problem. We’ve set it to auto-scale, so we’ll have all the instances we need.” But is everything all good?

4 min IT Ops

Log Data For Business Insights: How Marketing at Logentries Uses Logentries

Here at Logentries we’re dedicated to making log data simply accessible. So dedicated, in fact, that we eat our own dog food across every department. Our Co-Founder, Dr. Trevor Parsons, recently blogged about How To Track Business Metrics with Logs in 3 Simple Steps [/2013/10/how-to-track-business-metrics-in-3-simple-steps-log-em-tag-em-report-em/] . But I wanted to show how less technical people use Logentries within the company. While I have years of experience in the tech industry at various

4 min IT Ops

How To Track Peak Load and Memory Usage vs Response Time on Heroku

A few months back Heroku introduced log-runtime-metrics, which you can enable [https://devcenter.heroku.com/articles/log-runtime-metrics#enabling] from the command line to insert CPU load and memory usage metrics into your log events at 20 second intervals. Like all log data in its raw format it’s not massively useful to see metrics getting dumped into your logs every 20 seconds. That’s not exactly what Heroku had in mind, however. At the same time they introduced log-runtime-metrics, Heroku al

5 min IT Ops

How To Receive Log Alerts Via Flashing Lights In Your Office or Home

This is a guest blog post by Jason Ruane, the technical director atMoposa [http://www.moposa.com/], a place for brides and grooms to plan and manage their wedding. In this post Jason describes how he used a Wi-Fi enabled light and Logentries alerts to receive Logentries alerts via flashing lights in his house. Jason and his team are long time users of Logentries, analyzing all their logs from multiple servers in one centralized, cloud location. How I receive my Logentries alerts via home lighti

4 min IT Ops

How To Track Business Metrics in 3 Simple Steps: Log 'em, Tag 'em, Report 'em

This post covers how to begin tracking business metrics in three simple steps, without the need for any complex event tracking in your code, instrumentation, or metrics database. Step 1: Log your metrics The first thing you need to do is to log events you want to track. Logging your data rather than writing events you want to track directly to a database has a number of advantages as highlighted in separate posts by a lot of different folks of late – e.g. see Wanelo’s approach tobusiness metri

2 min IT Ops

Two computer nerds enter a bar ....

This week Logentries announced the closing of our Series A investment of $10m from our investors (and more importantly our partners) Polaris, Floodgate, Frontline and RRE, and the addition of Andrew Burton as CEO to our team. However, it wasn’t long ago that Viliam (my co-founder; photo of us below) and I walked into Neary’s pub in the heart of Dublin to discuss the idea of democratizing log data… It was January of 2010, Viliam and I had both spent almost 10 years building system monitoring

2 min IT Ops

How to Log Client-side JavaScript Events – Logging for the Web with le.js

At Logentries we provide a comprehensive collection of client libraries [https://github.com/logentries]and inputs [https://logentries.com/doc/inputs/] which make it easy to dispatch log events from any tier in your existing infrastructure. Until now though, one platform has been left out, and it’s the most ubiquitous and widely-understood of all- the browser. Collecting events from the browser presents a number of challenges which have hindered the development of a viable solution: * There’s n

2 min IT Ops

Filtering file names on the client-side

The Logentries agent [https://logentries.com/doc/agent/] enables users to get set up very quickly.One of the reasons we provide it is for convenience, as using the agent is far easier than configuring similar tools like syslog [https://logentries.com/doc/syslog/]. To achieve that, the agent loads the latest configuration from Logentries on every start. Some users expressed concerns about this behavior. For example, should their account on Logentries be compromised, users had asked if one could

2 min IT Ops

Field-level search

Back in July we announced a substantial improvement to our search functionality, searching your log data with logical operators [/2013/07/searching-with-logical-operators/]. Today we are happy to announce another big step in improving our search facility. You can now perform ** field-level searches [https://logentries.com/doc/search/]**in Logentries. Field-level searches allow you to search for events where a particular field is equal to, less than or greater than a particular value and thus al

3 min IT Ops

Build your own SMS Alerts--Logentries and Clickatell Combined!

* [/content/images/le-img/2013/07/Moposa2.jpg] This is a guest blog post by Jason Ruane, the technical director at Moposa [http://www.moposa.com/], a place for brides and grooms to plan and manage their wedding. In this post Jason, describes how he used Logentries webhook alerts and Clickatell to receive Logentries alerts via SMS. Jason and his team are long time users of logentries, analyzing all their logs from multiple servers in one centralized, cloud location. How I receive my Logentries

3 min IT Ops

How to Best Structure your Logs: Log Analysis Tips and Best Practices with Gal Segal from eToro

This week we have a guest blog post by Gal Segal. Gal is an engineer at eToro [https://www.etoro.com/en/platforms/openbook/], the worlds Largest Social Trading & Investment Network. In this post Gal shares his thoughts on log analysis best practices including tips on how best to structure your logs as well as some useful patterns that can be applied within your log events. He also discusses how to use Logentries’ new log search [https://logentries.com/doc/search/] functionality to more easily

2 min IT Ops

Searching with Logical Operators

We’re proud to announce substantial improvements to our search functionality, providing you with a simple search language to perform powerful complex queries on your log events [https://logentries.com/doc/search/]. You can now use AND, OR, and NOT to search for different combinations of keywords. Dive into the full details of our new search features here >> [https://logentries.com/doc/search/] or read on for a quick overview. Logical Operators: AND, OR, NOT**** You can now use AND to search fo

3 min IT Ops

Customer Spotlight with Adept Mobile: Ever wonder how the New England Patriots/Miami Dolphins/Boston Celtics handle website demand capacity

We’re launching a new customer Q&A series, where we chat with our customers about how they’re using log data [https://logentries.com/about-us/], their technology stack, and their overall industry insights. We love talking to our users, because we always learn something new – and we’d like to share those insights with you. In our first interview, we caught up with Craig Heneveld, Director of Technology at Adept Mobile in Boston, to talk about the development challenges and successes of bringing

3 min IT Ops

Musings from an AWS Meetup (Boston)

After opening up our new Boston office earlier this year (for any of you locals we’re down in the innovation district on Summer St) we finally got the chance to attend out first AWS Boston meetup [http://www.meetup.com/The-Boston-Amazon-Web-Services-Meetup-Group/]. And we are quickly reminded that we should have attended sooner with some great talks on this month’s topic of ‘Big Data’ and a good turn out from the Boston AWS community. For anyone who missed it here’s a brief run down of what was

2 min IT Ops

Dev Ops - Scaling infrastructure

The theme of last Thursday’s DevOps Ireland talks at Engine Yard Dublin was automated deployment. For anyone who missed it – the first talk was by Ulrich Dangel (@mr_ud [https://twitter.com/mr_ud]) on using Grml [http://grml.org/]. The second talk was by Stéfan from Facebook on Kobold which they use to bring up and tear down clusters. Ulrich’s creative slides for the talk can be found here [http://dangel.im/presentations/Automated%20deployment%20using%20Grml.pdf]. In Logentries we mainly dea

2 min IT Ops

Heroku Account Consolidation-Single View of all your Logs

If you host multiple apps on Heroku, you know the pain of having to log in to a separate add-on account for each of your Heroku apps. Whether you’re monitoring several different production applications, have separate apps for your production, staging, and test environments, or are a consultant in charge of administering separate applications for each of your clients you know how irritating it can be to constantly have to switch between accounts. This is particularly annoying when you’re trying

1 min IT Ops

Logentries New Feature Round Up

Here at Logentries we’ve been hard at work over the last couple months developing great new features that make it easier than ever to get actionable insights from your system data [https://logentries.com]. We’ve already blogged about all of these features in the past as we’ve released them, but in case you’ve missed them here’s a quick roundup on what we’ve released in the past couple months as well as one more new feature that we haven’t had the chance to blog about. Alert Emails Now Link back

2 min IT Ops

April NodeJs MeetUp in Dublin - from a newbie's perspective.

As usual there was a good turn out at the monthly Node.js Dublin event in Engine Yard recently. Although some of the Logentries team have been working with Node.js for some time I have to admit I am a bit of a newbie… I’ve been interested in learning more about node in general as I constantly see it used for more and more cool use cases. I’ve provided a brief summary of my take on it at the April Node.js event for anyone who missed out – note, this is from a newbies perspective 🙂 Dominic Tarr

2 min IT Ops

Tail -f is dead, long live tail -f

We’ve all been there – test some new code, tail your logs, test some more new code, tail the logs some more… But lets face it with so many cool SAAS solutions for managing your systems [http://mir.aculo.us/2013/02/16/running-a-saas-here-are-some-services-youll-find-useful/] who want’s to be hanging around on the command line these days??? Ok I’m being facetious and I know the command line is still as powerful as ever, however you really shouldn’t have to switch from your logging solution [https

1 min IT Ops

Switching between UTC and local time

All Logentries servers are configured for the UTC timezone. We use this as the default timezone for all internal data including customer’s logs. However, at the browser UI level we detect a user’s local timezone and present all dates in a more human friendly way, i.e. in the local time. We now allow users to change their time zone to UTC. This can be very handy when you manage systems in multiple regions or have a distributed development team where you want to have a common timezone that you ca

1 min IT Ops

Live feed, now in beta!

We are happy to announce that we extended our UI with real-time view of incoming log entries [https://logentries.com/product/]. You’ll find this feature – commonly called as tailing – useful especially as you debug the code or track particular transactions. Our live feed is integrated within current Entries tab. You can work in three simple modes – History, Live, and Paused. [/content/images/le-img/2013/05/controls_history.png] In History mode, you work as usual. Search is performed over the

1 min IT Ops

IP address resolving now optional

This is a neat feature. Whenever we see an IP address in your logs, we do automatic IP resolving so that you see immeadiately who is behind it. This is optional now, toggable with a button in the header. Enjoy! [/content/images/le-img/2013/04/ip_resolving.png]

3 min IT Ops

Inaugural PHP Dublin Meetup at Engineyard

I went along to the inaugural PHP meetup in Engineyard last Tuesday in the hope of PHP enlightenment. PHP wouldn’t be my goto language for most things but it does seem to have a very large following and a vibrant community and I was sure I would get something out of these talks. Realtime communication using PHP, Redis and socket.io The first talk of the night was by David Clay Smith [https://twitter.com/dclay] about doing real time communication for PHP applications. The typical use cases wer

1 min IT Ops

Heroku's Add-On Provider Summit - London

We’re just back from Heroku’s Add-on Provider Summit in London where we were getting the skinny on Heroku’s upcoming addon provider roadmap! [/content/images/le-img/2013/04/london.jpg] We were also stoked to spend time with our fellow addon providers from this side of the pond (includingJustOneDB [http://www.justonedb.com/], Cloudmailin [http://www.cloudmailin.com/], Pusher [http://pusher.com/], Nexmo [https://www.nexmo.com/], & Sharedworkforce [http://www.sharedworkforce.com/]to name but a fe

2 min IT Ops

Logs as Data... Logs as SMS... Logentries & Twillio combined!

More and more frequently I’m seeing cool use cases of Log data. Using ‘logs as data’ is the concept of writing semi-structured data to your logs which can then in turn be consumed by other services that can put the data to good use e.g. creating dashbaords, autoscaling, anomaly detection etc.Check out the recent Heroku article [https://blog.heroku.com/archives/2013/3/19/log2viz]on this to see a whole list of tools and apps that are helping to drive this movement.Some great tools I’ve come acros

1 min IT Ops

Per-log retention period

Typically, you would like to keep logs from development environment (with all debugging messages enabled) for only a limited amount of time, while production logs far longer. Up to now you had to set the retention period for the whole account, keeping development logs longer than needed. We are happy to announce per-log retention configuration! It gives you the option to fine-tune your retention policy in a more fine-grained manner than with a default per-account setting. To set a new log retent

3 min IT Ops

Webhooks are HMAC authenticated!

At Logentries we provide alerting that allows users to get notified about important events. Alerts are sent via email, notifications on iPhone, and via webhooks. Webhooks are ideal for situations where you want to kick off an automated response to an incident or event. Typical examples include application restarts, automatic scaling… and as you well know, the list goes on! Since webhooks, and thus your automated reactions to events, go about their business without a human being in the loop it

1 min IT Ops

How to extract any archive

How to extract any archive in bash? This small snippet will help you. It takes care of spaces and hyphens in the archive name (and most solutions you find on the web are wrong in this matter). extract () { case "$1" in *.ar) ar x -- "$1" ;; *.tar) tar xf "$1" ;; *.tar.bz2|*.tbz2) tar xjf "$1" ;; *.tar.gz|*.tgz) tar xzf "$1" ;; *.tar.xz) tar xJf "$1" ;; *.rar) unrar x -- "$1" ;; *.zip) unzip ./"$1" ;;

2 min IT Ops

Getting terminal colors right

As a part of our work on ANSI escape code coloring, I looked in detail at default colors used in different command line terminals. It appears form the Wikipedia article [https://en.wikipedia.org/wiki/ANSI_escape_code] that colors are set at their brightest level with minor variances across implementations: [/content/images/le-img/2012/11/pals.png] Adapting these color schemes gives the result as in the following picture: [/content/images/le-img/2012/11/lumi0.png] If you try to read the te

1 min IT Ops

Let's have more colors

PLEASE NOTE: The functionality described in this post was deprecated in a later version of Logentries. Are you a color lover? Then we have great news for you – we are interpreting ANSI escape codes to make your logging life more colorful! We do so by interpreting SGR parameters inside the event message body. ANSI escape codes (their SGR subset in particular) are embedded in text to modify its appearance such as color, style, and weight. It works the same way as tags in HTML, except there is no

1 min IT Ops

Feature requests

There are so many thing we can do with logs. We have a pile of new ideas on how to make Logentries better and we spend time implementing these every single day. We also collect feature requests from you, our lovely users. But we want to do more – we want to include you to the whole process of future development. We are happy to announce our Feature requests page, a page that enables you to add new feature requests, vote on them, and add comments with more details. Access the page from the sideb

2 min IT Ops

Cooking up Engine Yard logs with Chef

Opscode’s Chef is an open-source automation framework for the cloud, involving ‘recipes’ that can be triggered by events such as servers booting or restarting. Chef scripts are written in Ruby, but will often use OS-specific system calls to manipulate server instances. Chef is a great way to automate your cloud, and in particular it’s useful as your environment grows! On Engine Yard, Chef recipes can also be used to configure add-ons (like Logentries) on your servers, as we will explain… Each s

0 min IT Ops

Focus on the first input element in jQuery

Just in case you want to focus on the first element in a form, here is how to do it through one simple jQuery sector: $('#form-id :input:enabled:visible:first').focus(); Once you select the right form (#form-id), it’s time to specify any input elements including text areas (:input) which are editable (:enabled), visible (:visible), and first in the form (:first).

4 min IT Ops

How do I know if my Heroku app is slow?

So you’ve written a nice new Heroku app and have tested it thoroughly – it seems really fast – yay!!! And what do I mean by thoroughly… so you tried out all the functionality, everything works as you’ve expected, and the response time seems A-ok! Well at least it seems ok when there’s one request at a time…. but you wonder what happens when you go live and the floodgates open… what is the response time going to be like then and how will you know what your users are experiencing? Thanks for Swe

7 min IT Ops

Logentries goes Jelastic!

We are delighted to announce we have just teamed up with the guys at Jelastic to provide Logentries support for Jelastic users …. and to be perfectly honest we think its totally Jelastic! 😉 Our friends at Jelastic are also excited, so much so they’ve blogged about it too [http://blog.jelastic.com/2012/07/19/logging-on-jelastic-just-got-better-with-logentries/] ! With Jelastic’s next generation Java cloud platform, configuring enterprise Java applications ready for production, has been made dea

1 min IT Ops

Direct downloads

We are happy to announce a publicly available beta of direct downloads. Now you can download any part of your log stream with literally one click! We have provided a new button which you can see on the right side of the Log screen. Click on the download icon to start the download immediately. [/content/images/le-img/2012/06/download.png] You can configure different download options also. Specify whether you want to download log entries in plain text or if they should be compressed first. You

1 min IT Ops

iPhone notifications

If something is seriously broken in your application, you want to know about it immediately. But you can’t study your logs all the time – you have better thing to do (like coding right…). That’s why Logentries provides real-time alerts through both email and HTTP. And by real-time, we mean ‘real-time’ not 15 minutes later. Today, we bring real-time to a completely new level. We are happy to announce the availability of our iPhone application for receiving Logentries alerts [https://logentries.

1 min IT Ops

Logs in the (PHP)Fog

We are delighted to announce our recent integration on APPFog’s Rock Solid PHP cloud platform i.e. PHPFog! Right now logging integration is available to those on dedicated servers only (i.e. paid accounts) but we will be making Logentries available to all through PHPFog’s addon program very soon. We’ve been working with the guys at APPFog to make configuration dead simple. Simply go to our PHPFog setup page [https://logentries.com/doc/appfog/] and provide your email to create a new account.

3 min IT Ops

Collecting Heroku exception stack traces

On the Heroku platform, all error messages dumped on the console are acquired and logged line-by-line. While this is simple and straightforward, it has one unfortunate result – events that are spread on multiple lines appears as multiple events. This can be especially annoying for dumped exception stack traces. Typically, stack trace looks like this (note the interleaving router event): 215 1 2012-04-08T10:02:33+00:00 d. app web.2 - - /app/vendor/bundle/ruby/1.9.1/gems/eventmachine-0.12.10/lib

4 min IT Ops

How often should you look at your event and system logs? Daily, weekly, or just when there is a problem?

The motivation for this post came from a question on a Linkedin system administrator group [http://www.linkedin.com/groups?home=&gid=51443&trk=anet_ug_hm] this weekend, where the question was being asked: “Do you look at your event and system logs daily, weekly, or just when there is a problem?” And I guess a natural follow on question is: ***“and how often should you look at them?”. *** There’s no single answer that’s absolutely correct. When I’m asked this, I usually respond by saying som

8 min IT Ops

Logging on Linux (part 1...): I've got a bunch of linux boxes, what should I be looking out for in my logs?

While people generally understand that it is important to store and manage logs so that they can be regularly reviewed for operations, root cause analysis, security purposes etc… Oftentimes we find users struggle with figuring out where to start when reviewing their log data, and what they should be looking out for. A common question we get is: “ok so I’ve a bunch of linux boxes, I’m running a LAMP stack, what should I be looking out for in my logs”. Unfortunately, this is a little like trying

2 min IT Ops

Real-time alerts upgraded

Our alerting mechanism just got upgraded. Logentries has supported simple real-time alerts for some time now, however they were rather simplistic and while they served a need for our users we wanted to make them even more useful. We’ve now enhanced alerting with user-configurable limitations. So here’s how it works… With the new option It must match at least you can specify how many times the pattern MUST match in order to trigger the alert. The most common option Once triggers the alert on e

3 min IT Ops

Don't let your Heroku apps fail silently....

**Heroku error codes we’re coming to get you!!!! ** Logentries has come out of private beta on Heroku and is now available to all as a Heroku add on. Logentries provides log managenemt as a service with in built Heroku intelligence, so your applications don’t fail silently. [https://addons.heroku.com/logentries] “Ok, so why should I care?” Well, with the standard Heroku logging you get access to only the last 500 events, which is not great if you need to go and look at an issue that a custo

7 min IT Ops

Digging into Engine Yard Logs

I’ve recently been playing with a number of PAAS platforms, and its bringing me back somewhat to my days toying with J2EE application servers, JDBC drivers, Relational DBs etc. Oh how I remember deploying servers and databases and then checking out my shiny new application, remember the J2EE petstore [http://java.sun.com/developer/releases/petstore/] anyone?? 🙂 However the big difference with PAAS, over old school application servers is that you do not need to spend a few days configuring them

3 min IT Ops

Thoughts on Data Security

Right from day 1 of building our service, we were wondering how people would react to the idea of sending their logs to a third-party service. Would they trust the concept and would they trust us in the first place? However this is becoming less and less of an issue for our prospective customers and we find questions around ‘cloud security’ are raised a lot less often than when we first set out in 2010. Through 2011 it seems the market has become a lot more accepting of software as service. Sti

2 min IT Ops

Introducing Alerts

We are happy to introduce real-time alerts. And no need for third party integration – alerts are fully supported by Logentries. Use alerts to get informed about the events of interest in your log data via email or use our web hooks feature to integrate alerts with your systems. For example, configure an alert to email you for every unhandled exception in your app. Don’t fail silently, get alerted on error messages that matter so that you can react and keep your systems running as they should. A

1 min IT Ops

Logentries now available on AppHarbor (.NET as a Service)

We’re happy to announce Logentries is now available as an add-on service on AppHarbor [https://logentries.com/doc/appharbor]. [http://www.appharbor.com/addon] AppHarbor is a .NET Platform-as-a-Service. It makes deployment of .NET application easier and more fluent than has traditionally been the case. Azure done right as they say. Developers push code to AppHarbor using either Git or Mercurial. AppHarbor then builds the code and runs any unit tests. If everything checks out, the code is depl

0 min IT Ops

CloudArena & Business Intelligence in the Cloud

We recently presented at CloudArena [http://www.cloud-arena.com/business-intelligence-and-analytics-in-the-cloud.html] , where Business Intelligence was the topic of the day. A great event with a good mix of industry and academia and presentations outlining how we can extract more knowledge from large volumes of data. Naturally we spoke about log management and why it is particularly important for Cloud based systems. The CloudArena guys also put together a short clip capturing the thoughts of

1 min IT Ops

Logentries gets PM approval @dogpatchlabs

The Irish Taoiseach (Prime Minister) Enda Kenny, dropped by the new Dogpatch Dublin Labs recently to give Logentries a prime ministerial thumbs up. We didn’t manage to get the Taoiseach’s credit card details, but he did promise to sign up for a free account 😉 An Taoiseach Enda Kenny, meeting the Logentries team. The Logentries team along with some other very exciting companies (including Profitero, Boxever, Scrazzel, Hypertiny, Notrollo, Intercom,

1 min IT Ops

Logentries, now an Engine Yard partner

News just in… well last week in fact … Engine Yard have announced their new Engine Yards Platform Services [http://www.cloudave.com/15037/paas-is-the-future-of-cloud-services-engine-yard-opens-up-platform-services/] , a partner program offering access to add-on cloud services on the Engine Yard platform. @Logentries we are busily building our integrated service on Engine Yard, so if you are a Ruby developer or already an Engine Yard customer, you’ll be able to seamlessly avail of Logentries on E

3 min IT Ops

Is my mobile app buggy or do they simply not like it????

So you’ve just developed a new mobile application. Everything looks fine when you test in-house… yet when you release the application, your users are not so keen, and your application ratings are low…. hmmmmm ….you ask yourself “is the application buggy or do they simply not like it??” This can be difficult to ascertain without having access to log data from your mobile apps as they run in the field. Can you be sure that last fix didn’t introduce some new problem?? You can always go and check

2 min IT Ops

Ruby Ireland August Extraveganza - Review

Thanks to all the folks who showed up at the recent Ruby Ireland event [http://rubyireland.com/ruby-ireland-august-extravaganza/] and thanks to Engineyard [http://www.engineyard.com] for co-sponsoring some drinks with us that helped to warm up the crowd 🙂 Eamon Leonard of the recently acquired Orchestra [http://techcrunch.com/2011/08/23/engine-yard-acquires-orchestra-to-add-php-support-to-its-paas/] was first to speak. Orchestra is a platform for deploying, scaling and managing PHP applicati

1 min IT Ops

Logentries - try it now for free - no signup required!

We’ve been busily working away on our new site (we hope you like it). One of the main challenges we have faced to date is configuring users logs with our service. While this can be as simple as a one line configuration in your syslog config file, if you do not know your way around it can be a little confusing. We’ve found that once people get their logs into Logentries they generally love it and stick with it – so we’ve decided to make it even easier to get configured! To quote Dean MCCleland,

1 min IT Ops

Its node joke... Node.js now supported @Logentries

Our friends at chartaca.com [http://chartaca.com] have developed a nodje.js Logentries library [https://github.com/rjrodger/node-logentries] allowing you to make use of the node.js event driven model to log directly to your logentries account – thanks Richard [https://github.com/rjrodger]!! Whats’s this Node.js I hear you say ??? See Felix Geisendörfer’s post Understanding node.js [http://debuggable.com/posts/understanding-node-js:4bd98440-45e4-4a9a-8ef7-0f7ecbdd56cb] for some background expl

1 min IT Ops

Logging ..... It's a Gem!!!

If you’re a lover of precious stones, in particular those of the blood red, ruby kind, you’ll be glad to know Logentries, log management as a service, is now easily configurable with your Ruby applications. Logentries allows you to send your log data directly from your source code to the Logentries API. Logentries supports TCP, UDP, HTTP PUT, HTTPS PUT on api.logentries.com. However, our new Ruby logging library [https://github.com/logentries/le_ruby] does the hard work for you. You can simply

3 min IT Ops

Caching AJAX Responses in JavaScript

Ajax is a great tool for creating responsive dynamic web pages. Although Ajax updates are usually much faster than page reloads, there is still considerable delay when considering Ajax for pages that require real-time responses (content updating in real time). This gets even more complicated in fully dynamic user interfaces. Interfaces structured within web pages with URLs that allow for anchor and back button navigation and which require real time updating, in particular, can suffer from repea

3 min IT Ops

Strictly HTTPS

Default HTTP communication is not encrypted. That is mostly fine for ‘normal’ web browsing. Encryption was considered only for highly sensitive communication such as managing bank accounts. Common man-in-the-middle attacks were well known, but often they were thought of as in a theoretical sense and not really feared as accessing a ‘wire’ can be physically difficult. We are less secure nowadays. With the increasing use of wireless networks the communication can be eavesdropped by literally anyo

1 min IT Ops

Log Management as a (Windows) Service

Windows support has been added as part of our latest release. You can now: * monitor logs from any applications running on Windows * monitor Windows event logs * configure the Logentries agent as a Windows service The Logentries agent comes in the form python script or as a Windows executable. The executable is useful if you don’t want to download Python for windows and the Python/Windows extensions. Configuring Python on Windows may not be ideal if you have lots and lot of servers that you

3 min IT Ops

CSS Color Picker

As a part of our work to make user-defined tags accessible, we have used a color picker to allow users to select appropriate colors for the different tags they create. After looking around at some existing color pickers, we could not find any suitable implementations. Our requirements were something which could be both usable and visually appealing. So we ended up with our own… As a first step, we searched for a slick looking visual color picker which would allow for quick color selection. The

1 min IT Ops

User-Defined Tags

Using the logentries Tags feature you can visualize and categorize specified events in the log graphs with colored indicators. This feature also allows you to display these events in a filtered list according to the specified category (e.g. fatal error, critical event, exception). We are happy to announce an extension to this popular feature: user-defined tags . This enhancement allows you to define custom tags with a unique visual appearance, i.e. you can select your own colored indicators and

2 min IT Ops

Stronger Passwords for Django

One of our main concerns is data security. While we can do our best to protect our service against external threats, a weak account password posses the easiest attack vector. We are all human and sometimes we don’t even realize how vulnerable our (supposedly strong) password is to a dictionary-based attack. We use Django [http://www.djangoproject.com/] internally. Let us share with you how we hard-ended our account registration process to automatically check for weak passwords and give our user

1 min IT Ops

Tagging

We have recently introduced a nice feature – tagging. A tag is a small piece of information you can pinpoint to the event to highlight its meaning and severity. The tag is displayed in graphs and enables easy break down to different event categories. The easiest and immediate use is to shows errors. Bold colored marks give you immediate navigation to “interesting” events. Here is how it looks in the graph. Those red circles clearly shows where those errors are: [/content/images/le-img/2011/02

Quick Cookie Notification

Posts tagged Logentries

R7-2017-18: Logentries Windows Agent uses vulnerable OpenSSL (FIXED)

Network Administrator’s Guide to Surviving an Audit: During the Audit

Logging in a Software Defined Network

Habits that Pay Off for Programmers

Legacy apps in the data center, today's apps in the cloud

Network Administrator’s Guide to Surviving an Audit: Preparation

Log Analysis for System Troubleshooting

Logging for Fun: Things You'd Never Thought to Log

Roots and Culture: Logging and the Telephone Bill

Never type a search query again!

Finalists in FIVE categories at the Network Computing Awards!

5 Rules of Pair Programming Etiquette

3 Predictions for Development in the Age of Serverless Computing

Java 8 - Lazy argument evaluation

12 Days of HaXmas: The Gift of Endpoint Visibility and Log Analytics

Logs To Understand User Activity and Behavior

The Value of Correlation IDs

The Generosity of Thought: Caring and Sharing in the Open Source Community

Solving the expression problem

Announcing InsightOps - Pioneering Endpoint Visibility and Log Analytics

Node.js as a Proxy to Logentries.com

Goodbye to the VCR: Rewinding Down Memory Lane

Widely-used Android App Leaks MS Exchange Credentials

Top 5 Questions from the October Walkthrough Webinar

Empowering Leexportpy with new services

Overview of 'online' algorithm using Standard Deviation example

Logging OwnTracks to Logentries

Logging Mosquitto Server logs (from Raspberry Pi) to Logentries

Using Event and Log Data to Assure Security and Compliance

UNITED 2016: Want to share your experience?

Moving away from MVC

Avoid These Things When Logging From Your Application

Designing Data Access Object (DAO) Layer

Log Search Simplified

Get your work done even faster with the Logentries REST API

Hashing Infrastructures

Logging the Lights in Your Home

Using Log Data as Forensic Evidence

4 Potential Security Issues Raised By Pokémon Go

Nexpose Logging Analytics using LogEntries

3 Ways to Improve Your Code Testing

Integrating Logentries with OpsGenie: 3 Easy Steps

Exporting Logentries data with Leexportpy

What exactly is an Event-loop?

Introduction to the Logentries Command Line Interface

Using Logentries With Angular v1.5

Top 5 Questions from the July Walkthrough Webinar

REST API: a little cURL and some Python

Backup Log Checks and What They Can Tell You

Keep Your Code Clean while Logging

Rsyslog TCP/IP log forwarding to Logentries with Netscaler

Introducing LEQL: LIMIT

Introduction to Leexportpy

Self-describing Logging Using Log4J

How Audit Logs Help Confirm and Correct Security Policy

Signal AND Noise The Best of All Worlds for Logging

SSH from the ground up

Migrating a web app to Angular

Webinar Recap: Tableau Server Log Analytics

Monitoring SNS Activity Using a Lambda Function and Logentries

Checking Active Directory (AD) Security and Integrity via Log Monitoring

Exploiting Zookeeper for managing processes in a production environment with Lockex

Top 5 Questions from the June Walkthrough Webinar

Introducing LEQL: BYTES

A different way to log your website usage

Tableau Server Log Analytics: Easily Monitor Your Data Analytics Stack

Logging for Continuous Integration

Webinar Recap: How to Easily Send and Use your IoT Data in the Cloud

Integrating Logentries With .NET The How and Why

Intrinsically fast: more JVM performance tinkering

Now Available: REST Query API

Implementing Self-Describing Log Data Using NodeJS

How To: Send Logentries Alerts to BigPanda

Raspberry Pi, Logs and IoT - Sending Pi Log and Sensor data to Logentries

Webinar: How to Easily Send and Use your IoT Data in the Cloud

A Query Language for Your Logs

Integrating the Logentries Javascript Library With React

Logs for SEO

Q&A with Sysdig's Chris Crane + Upcoming Webinar

Getting started with Docker, Google Container Engine & Logentries