2 min
IT Ops
AWS CloudWatch Logs - Making Innovation Easier and Cheaper
Last week AWS made an exciting announcement at their NYC Summit, which I believe
is a big step in helping to reduce the cost of log management in the cloud
[https://logentries.com/doc/log-management/] for end users and vendors alike –
good news all around!
What was announced?
Amazon announced ‘CloudWatch Logs’ which essentially allows you to send log data
from your EC2 instances into CloudWatch for storage and some rudimentary
analysis.
How does it work?
CloudWatch logs works by collecting y
2 min
Logentries
Logs to Metrics to JSON to Geckoboard
The world of log management [https://logentries.com/doc/log-management/] and
application performance monitoring [/2014/04/apm-is-not-enough/]are evolving and
there is no longer a big distinction between the two.
Up to even a year ago, many people in IT viewed logs as simply a means of
identifying and tracking issues once they have occurred within their system. It
was rare that people were using their logs for more than basic application event
tracking and watching for exceptions.
But over the
3 min
IT Ops
TCP or UDP for Logging – Which One to Use?
When sending data over the Internet, we want to be sure that data has arrived
safely and is delivered in a reliable manner. Assuming that our hardware and
operating systems do the job well, we can use transport protocol on the
application level to achieve that goal. In theOSI
[http://en.wikipedia.org/wiki/OSI_model] model the two most populartransport
protocols [http://en.wikipedia.org/wiki/Transport_layer] areTCP
[http://en.wikipedia.org/wiki/Transmission_Control_Protocol] andUDP
[http://en.wik
2 min
IT Ops
Using Log Analysis to Find the .18%
Even the most non-technical audiences realize the business value in analyzing
their log data. DevOps professionals are constantly being asked to monitor their
application performance, and often rely on log data for troubleshooting,
diagnostics and application systems monitoring. The challenge is that the amount
of log data generated can be completely overwhelming.
Why log analysis is so difficult?
In one word — volume. Even relatively small applications, can generate millions,
or billions of lo
4 min
Logentries
How to Get Full Visibility of CloudTrail, CloudWatch & System Logs
At re:Invent last year
[/2013/11/reinvent-keynote-overview-amazon-cloudtrail-appstream-and-workspaces-announced/]
AWS launched CloudTrail [http://aws.amazon.com/cloudtrail/], which is
essentially a “trail” of all activity in your AWS Cloud environment… the clue is
in the name ;). CloudTrail has been widely accepted as a great addition to the
AWS toolset as it is useful for numerous use cases including security analysis,
troubleshooting, tracking of AWS resource usage, and more.
While CloudTrai
4 min
IT Ops
Google Cloud DataFlow – A Game Changer?
So the biggest revolution in database and analytics technology – namely the
distributed batch processing technique known as MapReduce (and the associated
Hadoop-centric ecosystem that has built up around it) is a legacy technology
[http://venturebeat.com/2014/06/25/google-cloud-dataflow/] for one Silicon
Valley player. Last week Google announced the arrival of Google Cloud Dataflow –
a new service for cloud-based big data analytics that, Google says, supersedes
MapReduce.[
While various VCs and
3 min
IT Ops
Smart Integration Testing with Dropwizard, Flyway and Retrofit
It’s widely understood that increasing the scope and complexity of a piece of
software almost always dramatically increases the effort required to verify it.
Verification typically entails testing the behavior of the new feature, plus
ensuring no existing functionality has been adversely affected.
Because this kind of testing can quickly become painful for even the simplest of
components, and because at Logentries
[https://logentries.com/centralize-log-data-automatically/?le_trial=dropwizard_f
3 min
IT Ops
Why You Should Analyze Your Logs When Load Testing
We recently hosted awebinar about running load tests
[/2014/06/how-to-run-jmeter-tests-for-500k-users-analyze-log-data-results/]
(with BlazeMeter [http://www.blazemeter.com]) and then analyzing the log data
generated on the application under test (with Logentries [http://logentries.com]
). We had a great response and I wanted to share a few more thoughts about why
load and stress testing your applications is an important step in understanding
how your systems will behave in production when incr
1 min
IT Ops
How to Run JMeter Tests for 500k Users & Analyze Log Data Results
Logentries recently co-hosted a webinar with BlazeMeter [http://blazemeter.com]
on advanced JMeter testing for 500,000+ users. The webinar had almost 1,000
registrants, and focused on helping users understand what’s happening on
their systems during load testing; identify any warnings or exceptions being
thrown; and how to use log data to analyze performance under this load level.
In this webinar, Logentries co-founder Trevor Parsons and BlazeMeter DevOps Team
Leader Refael Botbol covered how
3 min
IT Ops
Correlating Logs and Server Metrics...Finally!
Server resource usage information is a key data point that is critical for any
Operations/DevOps team. I like to think of server resource usage information as
analogous to the dashboard of your car — it can tell you if you are running out
of gas or if something is overheating your system. Similarly, if you keep a
record of your server metrics you can also use them to investigate and debug
issues in your systems, understand performance trends as well as use in capacity
planning over time.
[/con
4 min
IT Ops
Debugging Rails with Logs
This article was written by Jack Kinsella, a Logentries customer based in
Berlin. Jack owns Oxbridge Notes and co-founded Bolivian Express. Below Jack has
shared an excerpt from his writings on A Comprehensive Guide to Debugging Rails.
Seasoned Rails pros will have interpreting logs down second nature, but I’ll
outline the important realizations I had as a beginner for the sake of the
version of me out there who is new to Rails development. Whenever you visit a
page/fill a form/do an Ajax reque
2 min
IT Ops
Collecting and Analyzing Logs from Your Java Application
le_java is one of several libraries that allow you to send log data directly to
your Logentries account from your application. le_java supports both the log4j
and logback logging libraries in java, and can be configured to work with
either – in just a few steps! (Check out:
github.com/logentries/le_java/blob/master/README.md
[https://github.com/logentries/le_java/blob/master/README.md] for our complete
tutorial.)
[/content/images/le-img/2014/06/java.jpg]
Once you have your application send
2 min
IT Ops
Our Favorite Linux Performance Monitoring Tools
As a part of monitoring and troubleshooting our system and applications, we
often need to get a quick snapshot of information about the status of our
server. This usually means SSHing into a server and running certain commands to
get to the information we need. I wanted to share a quick overview of my top 5
commands to get a snapshot of this server information.
htop [http://hisham.hm/htop/]
htop is an ncurses-based interactive process viewer for Linux. It gives
information about the CPU utiliz
2 min
IT Ops
Using D3.js to Graph Your Log Data
At Logentries, we use the open-source D3.js visualization library for a number
of our graphs, including our recently released Insights feature
[/2014/05/create-dashboards-from-your-logs-data/]. In a nutshell, D3 allows you
to efficiently manipulate documents based on data with minimal overhead. While
it could in fact be used for all types of DOM manipulation that you might do
with jQuery instead for example, we have used it purely for its graphing
functionality.
One of the reasons we chose D3 i
2 min
IT Ops
Log Aggregation & Grouping in 3 Clicks
With the introduction of a centralized,aggregated view of your logs
[https://logentries.com/doc/ui/#Aggregated_View], we enabled our users to see
their entire stack in one view and to quickly correlate different logs together.
This log aggregation viewpoint provides much deeper insight into what is
occurring across your logs from various sources.
We thought that the Log Aggregation View was pretty cool, so we are very excited
to launch log Groups which add even more flexibility to save and moni