Welcome to the world of Cross-Site Scripting (XSS) vulnerabilities—one of the most common and dangerous security issues in frontend development.

Why XSS Should Keep You Up at Night

XSS isn't just a theoretical security concern that happens to "other people's apps." It's a real threat that can:

• Steal user session tokens and authentication cookies

• Redirect users to malicious websites

• Modify the DOM to create fake login forms

• Access sensitive user data and send it to attackers

• Turn your application into a launching pad for attacks on other users

The scary part? Most XSS vulnerabilities in modern frontend applications don't come from obvious places like eval() or innerHTML. They hide in seemingly innocent features like user profiles, comment systems, notification displays, and rich content editors.

As a frontend engineer, you're the first line of defense. Understanding XSS isn't just about passing interviews—it's about building applications that users can actually trust.

The Three Faces of XSS

Let's break down the three main types of XSS vulnerabilities you'll encounter in real-world frontend applications.

1. Stored XSS: The Persistent Threat

Stored XSS is like a time bomb. The malicious script gets saved to your database and executes every time someone views the infected content.

Here's how it typically happens in a React application:

// UserProfile.js - VULNERABLE CODE
import { useState, useEffect } from 'react';

function UserProfile({ userId }) {
  const [profile, setProfile] = useState(null);

  useEffect(() => {
    fetch(`/api/users/${userId}`)
      .then(res => res.json())
      .then(data => setProfile(data));
  }, [userId]);

  if (!profile) return <div>Loading...</div>;

  return (
    <div className="profile">
      <h2>{profile.name}</h2>
      <div className="bio">
        {/* DANGER: This will execute any script tags in the bio */}
        <div dangerouslySetInnerHTML={{ __html: profile.bio }} />
      </div>
    </div>
  );
}

How the attack works:

1. Attacker creates a profile with bio: <script>fetch('/api/steal-data', {method: 'POST', body: document.cookie})</script>

2. This gets stored in your database

3. Every time someone views this profile, the script executes

4. The attacker now has access to every visitor's session cookies

The fix:

// UserProfile.js - SECURE VERSION
import { useState, useEffect } from 'react';
import DOMPurify from 'dompurify';

function UserProfile({ userId }) {
  const [profile, setProfile] = useState(null);

  useEffect(() => {
    fetch(`/api/users/${userId}`)
      .then(res => res.json())
      .then(data => setProfile(data));
  }, [userId]);

  if (!profile) return <div>Loading...</div>;

  // Sanitize HTML before rendering
  const sanitizedBio = DOMPurify.sanitize(profile.bio, {
    ALLOWED_TAGS: ['p', 'br', 'strong', 'em', 'ul', 'ol', 'li'],
    ALLOWED_ATTR: []
  });

  return (
    <div className="profile">
      <h2>{profile.name}</h2>
      <div className="bio">
        <div dangerouslySetInnerHTML={{ __html: sanitizedBio }} />
      </div>
    </div>
  );
}

2. Reflected XSS: The Mirror Attack

Reflected XSS happens when user input is immediately reflected back in the response without proper sanitization. This is common in search results, error messages, and URL parameters.

Here's a vulnerable Next.js search page:

// pages/search.js - VULNERABLE CODE
import { useRouter } from 'next/router';

export default function SearchPage() {
  const router = useRouter();
  const { q } = router.query;

  return (
    <div>
      <h1>Search Results</h1>
      {q && (
        <p>
          {/* DANGER: URL parameter directly rendered */}
          You searched for: <span dangerouslySetInnerHTML={{ __html: q }} />
        </p>
      )}
      {/* Search results would go here */}
    </div>
  );
}

How the attack works:

1. Attacker crafts a malicious URL: https://yourapp.com/search?q=<script>alert('XSS')</script>

2. They trick users into clicking this link (via email, social media, etc.)

3. When users visit the link, the script executes immediately

The fix:

// pages/search.js - SECURE VERSION
import { useRouter } from 'next/router';

export default function SearchPage() {
  const router = useRouter();
  const { q } = router.query;

  return (
    <div>
      <h1>Search Results</h1>
      {q && (
        <p>
          {/* React automatically escapes this content */}
          You searched for: <span>{q}</span>
        </p>
      )}
      {/* Search results would go here */}
    </div>
  );
}

The key insight here: React's default behavior is to escape content, which protects you. The vulnerability only appears when you deliberately bypass this protection with dangerouslySetInnerHTML.

3. DOM-based XSS: The Client-Side Trap

DOM-based XSS happens entirely in the browser, without the server being involved. The malicious payload is processed by client-side JavaScript.

Here's a common pattern in React apps that handle URL fragments:

// components/WelcomeMessage.js - VULNERABLE CODE
import { useEffect, useState } from 'react';

function WelcomeMessage() {
  const [message, setMessage] = useState('');

  useEffect(() => {
    // Getting user name from URL fragment
    const hash = window.location.hash;
    if (hash.startsWith('#welcome=')) {
      const userName = hash.substring(9); // Remove '#welcome='
      // DANGER: Directly inserting URL content into DOM
      document.getElementById('welcome').innerHTML = `Welcome back, ${userName}!`;
    }
  }, []);

  return <div id="welcome"></div>;
}

How the attack works:

1. Attacker creates a malicious URL: https://yourapp.com/#welcome=<img src=x onerror=alert('XSS')>

2. The fragment is processed by client-side JavaScript

3. The malicious code executes without any server interaction

The fix:

// components/WelcomeMessage.js - SECURE VERSION
import { useEffect, useState } from 'react';

function WelcomeMessage() {
  const [userName, setUserName] = useState('');

  useEffect(() => {
    const hash = window.location.hash;
    if (hash.startsWith('#welcome=')) {
      const name = hash.substring(9);
      // Let React handle the escaping
      setUserName(decodeURIComponent(name));
    }
  }, []);

  return (
    <div>
      {userName && <p>Welcome back, {userName}!</p>}
    </div>
  );
}

XSS in Unexpected Places

Now that you understand the basics, let's explore where XSS vulnerabilities love to hide in modern SaaS applications.

Content Management Systems

Many SaaS apps have marketing pages managed through a CMS. Here's a vulnerable pattern:

// components/MarketingBanner.js - VULNERABLE CODE
import { useState, useEffect } from 'react';

function MarketingBanner() {
  const [bannerContent, setBannerContent] = useState('');

  useEffect(() => {
    fetch('/api/cms/banner')
      .then(res => res.json())
      .then(data => setBannerContent(data.html));
  }, []);

  return (
    <div 
      className="marketing-banner"
      dangerouslySetInnerHTML={{ __html: bannerContent }}
    />
  );
}

The problem: If your CMS doesn't properly sanitize content, or if an attacker gains access to your CMS, they can inject malicious scripts that will run on every page.

Markdown Rendering

Rich text editors and markdown processors are common XSS entry points:

// components/BlogPost.js - VULNERABLE CODE
import { marked } from 'marked';

function BlogPost({ content }) {
  const htmlContent = marked(content);

  return (
    <article 
      dangerouslySetInnerHTML={{ __html: htmlContent }}
    />
  );
}

The secure approach:

// components/BlogPost.js - SECURE VERSION
import { marked } from 'marked';
import DOMPurify from 'dompurify';

function BlogPost({ content }) {
  const htmlContent = marked(content);
  const sanitizedContent = DOMPurify.sanitize(htmlContent, {
    ALLOWED_TAGS: ['p', 'br', 'strong', 'em', 'h1', 'h2', 'h3', 'ul', 'ol', 'li', 'a', 'code', 'pre'],
    ALLOWED_ATTR: ['href', 'title'],
    ALLOWED_SCHEMES: ['http', 'https', 'mailto']
  });

  return (
    <article 
      dangerouslySetInnerHTML={{ __html: sanitizedContent }}
    />
  );
}

Notifications and Toasts

Notification systems often display user-generated content, making them prime XSS targets:

// components/NotificationToast.js - VULNERABLE CODE
function NotificationToast({ notification }) {
  return (
    <div className="toast">
      <div 
        className="toast-message"
        dangerouslySetInnerHTML={{ __html: notification.message }}
      />
    </div>
  );
}

The secure version:

// components/NotificationToast.js - SECURE VERSION
function NotificationToast({ notification }) {
  return (
    <div className="toast">
      <div className="toast-message">
        {notification.message}
      </div>
    </div>
  );
}

Safely Rendering User-Generated Content

Let's dive deep into how to handle user-generated content (UGC) securely in React and Next.js applications.

Setting Up DOMPurify

First, install and configure DOMPurify:

npm install dompurify
npm install @types/dompurify  # If using TypeScript

Create a utility function for consistent sanitization:

// utils/sanitize.js
import DOMPurify from 'dompurify';

export const sanitizeHtml = (dirty, options = {}) => {
  const defaultOptions = {
    ALLOWED_TAGS: ['p', 'br', 'strong', 'em', 'ul', 'ol', 'li', 'a'],
    ALLOWED_ATTR: ['href', 'title', 'target'],
    ALLOWED_SCHEMES: ['http', 'https', 'mailto'],
    FORBID_TAGS: ['script', 'object', 'embed', 'base', 'link'],
    FORBID_ATTR: ['style', 'onerror', 'onload', 'onclick']
  };

  const mergedOptions = { ...defaultOptions, ...options };
  return DOMPurify.sanitize(dirty, mergedOptions);
};

// For different content types
export const sanitizeMarkdown = (dirty) => {
  return sanitizeHtml(dirty, {
    ALLOWED_TAGS: ['p', 'br', 'strong', 'em', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 
                   'ul', 'ol', 'li', 'a', 'code', 'pre', 'blockquote'],
    ALLOWED_ATTR: ['href', 'title']
  });
};

export const sanitizeBasicText = (dirty) => {
  return sanitizeHtml(dirty, {
    ALLOWED_TAGS: ['p', 'br', 'strong', 'em'],
    ALLOWED_ATTR: []
  });
};

Creating a Safe HTML Component

Build a reusable component for rendering sanitized HTML:

// components/SafeHtml.js
import { sanitizeHtml } from '../utils/sanitize';

function SafeHtml({ html, className = '', sanitizeOptions = {} }) {
  const cleanHtml = sanitizeHtml(html, sanitizeOptions);

  return (
    <div 
      className={className}
      dangerouslySetInnerHTML={{ __html: cleanHtml }}
    />
  );
}

export default SafeHtml;

Usage Examples

// components/UserComment.js
import SafeHtml from './SafeHtml';

function UserComment({ comment }) {
  return (
    <div className="comment">
      <div className="comment-author">{comment.author}</div>
      <SafeHtml 
        html={comment.content}
        className="comment-content"
        sanitizeOptions={{
          ALLOWED_TAGS: ['p', 'br', 'strong', 'em', 'a'],
          ALLOWED_ATTR: ['href']
        }}
      />
    </div>
  );
}

Security Best Practices for Frontend Teams

1. Avoid dangerouslySetInnerHTML When Possible

The best defense against XSS is to avoid rendering raw HTML altogether. React's default escaping behavior protects you:

// GOOD: React automatically escapes this
function UserName({ name }) {
  return <span>{name}</span>; // Safe even if name contains <script>
}

// RISKY: Only use when absolutely necessary
function RichContent({ html }) {
  return <div dangerouslySetInnerHTML={{ __html: sanitizeHtml(html) }} />;
}

2. Implement Content Security Policy (CSP)

Configure CSP headers in your Next.js application:

// next.config.js
const nextConfig = {
  async headers() {
    return [
      {
        source: '/(.*)',
        headers: [
          {
            key: 'Content-Security-Policy',
            value: [
              "default-src 'self'",
              "script-src 'self' 'unsafe-inline'", // Be more restrictive in production
              "style-src 'self' 'unsafe-inline'",
              "img-src 'self' data: https:",
              "font-src 'self'",
              "connect-src 'self'",
              "frame-ancestors 'none'"
            ].join('; ')
          }
        ]
      }
    ];
  }
};

module.exports = nextConfig;

For production, create a more restrictive CSP:

// production CSP
const productionCSP = [
  "default-src 'self'",
  "script-src 'self'", // Remove 'unsafe-inline'
  "style-src 'self'",
  "img-src 'self' data: https:",
  "font-src 'self'",
  "connect-src 'self' https://api.yourservice.com",
  "frame-ancestors 'none'",
  "base-uri 'self'",
  "form-action 'self'"
].join('; ');

3. Input Validation Strategies

Validate user input both on the client and server side:

// utils/validation.js
export const validateUserInput = (input, type = 'text') => {
  const validators = {
    text: (str) => typeof str === 'string' && str.length <= 1000,
    email: (str) => /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(str),
    url: (str) => {
      try {
        const url = new URL(str);
        return ['http:', 'https:'].includes(url.protocol);
      } catch {
        return false;
      }
    }
  };

  return validators[type] ? validators[type](input) : false;
};

// In your component
function CommentForm({ onSubmit }) {
  const [comment, setComment] = useState('');
  const [error, setError] = useState('');

  const handleSubmit = (e) => {
    e.preventDefault();

    if (!validateUserInput(comment, 'text')) {
      setError('Invalid comment content');
      return;
    }

    onSubmit(comment);
  };

  return (
    <form onSubmit={handleSubmit}>
      <textarea 
        value={comment}
        onChange={(e) => setComment(e.target.value)}
        maxLength={1000}
      />
      {error && <div className="error">{error}</div>}
      <button type="submit">Post Comment</button>
    </form>
  );
}

4. Safe URL Handling

Be careful when handling URLs from user input:

// utils/url.js
export const sanitizeUrl = (url) => {
  try {
    const parsed = new URL(url);
    // Only allow http and https protocols
    if (!['http:', 'https:'].includes(parsed.protocol)) {
      return null;
    }
    return parsed.toString();
  } catch {
    return null;
  }
};

// In your component
function LinkComponent({ href, children }) {
  const safeUrl = sanitizeUrl(href);

  if (!safeUrl) {
    return <span>{children}</span>;
  }

  return (
    <a 
      href={safeUrl} 
      target="_blank" 
      rel="noopener noreferrer"
    >
      {children}
    </a>
  );
}

Interview Insights: How Interviewers Test XSS Knowledge

Understanding XSS isn't just about writing secure code—it's also about demonstrating your security awareness in interviews. Here's how interviewers typically approach this topic and what they're looking for.

Common Interview Questions

Question 1: "Walk me through how you would implement a comment system for a blog application. What security considerations would you keep in mind?"

What they're testing: Your ability to identify XSS risks in a common feature and propose mitigation strategies.

Good answer structure:

1. Identify the XSS risk: "User comments are a classic XSS vector because they contain user-generated content"

2. Explain the attack scenario: "An attacker could submit a comment containing script tags that would execute when other users view the comment"

3. Propose specific solutions: "I would sanitize all user input using a library like DOMPurify, validate input lengths, and implement CSP headers"

4. Mention additional considerations: "I'd also implement rate limiting, content moderation, and ensure the backend validates and sanitizes data as well"

Question 2: "You're reviewing a React component that uses dangerouslySetInnerHTML to render user profiles. What red flags would you look for?"

What they're testing: Your code review skills and understanding of XSS vulnerabilities.

Good answer points:

• Check if the HTML content is being sanitized before rendering

• Look for direct insertion of user input without validation

• Verify if the content source is trusted

• Consider if dangerouslySetInnerHTML is necessary or if plain text rendering would suffice

• Examine the allowed HTML tags and attributes

Question 3: "How would you implement a rich text editor that allows users to format their content but prevents XSS attacks?"

What they're testing: Your ability to balance functionality with security.

Good answer structure:

1. Explain the allowlist approach: "I would define a strict allowlist of safe HTML tags and attributes"

2. Mention sanitization: "Use DOMPurify to sanitize content both on the client and server"

3. Discuss CSP: "Implement Content Security Policy to prevent inline scripts"

4. Consider alternatives: "For simpler use cases, consider markdown instead of raw HTML"

Question 4: "A user reports that clicking a link in your application executed unwanted JavaScript. How would you investigate and fix this?"

What they're testing: Your incident response approach and debugging skills.

Good answer structure:

1. Immediate response: "First, I'd determine the scope—is this affecting multiple users or just one?"

2. Investigation: "I'd check for reflected XSS in URL parameters, examine how we handle user-generated links, and review recent code changes"

3. Mitigation: "Implement proper URL validation, sanitize link targets, and add rel='noopener noreferrer' to external links"

4. Prevention: "Set up monitoring for suspicious JavaScript execution and implement stricter CSP"

What Interviewers Want to Hear

Security-first mindset: Show that you consider security implications from the beginning of feature development, not as an afterthought.

Practical knowledge: Demonstrate familiarity with real-world tools and libraries (DOMPurify, CSP, etc.) rather than just theoretical concepts.

Defense in depth: Explain how you'd implement multiple layers of protection rather than relying on a single solution.

User impact awareness: Show that you understand how XSS attacks affect real users and business outcomes.

Red Flags to Avoid

• Dismissing XSS as "just a frontend problem" (it requires full-stack thinking)

• Suggesting overly complex solutions when simple ones exist

• Not mentioning user input validation

• Ignoring the importance of server-side validation

• Focusing only on technical solutions without considering user experience

XSS Prevention Practices Checklist

Here's your comprehensive checklist for preventing XSS vulnerabilities in frontend development:

Development Phase

• [ ] Default to safe rendering: Use React's default text rendering instead of dangerouslySetInnerHTML whenever possible

• [ ] Sanitize all user input: Implement DOMPurify or similar sanitization library for any HTML content

• [ ] Validate URLs: Check user-provided URLs for safe protocols (http/https only)

• [ ] Use allowlists: Define strict allowlists for HTML tags and attributes rather than blocklists

• [ ] Implement CSP: Configure Content Security Policy headers to prevent inline script execution

• [ ] Validate on both ends: Implement validation on both client and server sides

Code Review Phase

• [ ] Audit dangerouslySetInnerHTML usage: Every instance should be reviewed and justified

• [ ] Check user input handling: Verify all user inputs are properly validated and sanitized

• [ ] Review third-party content: Ensure external content (APIs, CMSs) is treated as untrusted

• [ ] Examine URL handling: Check for proper validation of user-provided URLs and links

• [ ] Verify escape sequences: Ensure special characters are properly escaped in all contexts

Testing Phase

• [ ] Test with malicious payloads: Try common XSS payloads in all input fields

• [ ] Verify CSP effectiveness: Test that your CSP headers actually prevent script injection

• [ ] Check different browsers: Ensure XSS protection works across different browsers

• [ ] Test edge cases: Verify handling of empty inputs, very long inputs, and special characters

• [ ] Validate sanitization: Ensure your sanitization logic handles all expected HTML elements

Deployment Phase

• [ ] Configure production CSP: Set strict CSP headers for production environment

• [ ] Enable security headers: Implement additional security headers (X-Content-Type-Options, X-Frame-Options)

• [ ] Monitor for violations: Set up CSP violation reporting to detect potential attacks

• [ ] Regular security audits: Schedule periodic security reviews of user-facing features

• [ ] Keep dependencies updated: Regularly update sanitization libraries and security-related packages

Ongoing Maintenance

• [ ] Security training: Keep the team updated on latest XSS attack vectors and prevention techniques

• [ ] Incident response plan: Have a clear process for handling security incidents

• [ ] Documentation: Maintain clear guidelines for secure coding practices

• [ ] Penetration testing: Conduct regular security testing of your applications

Remember, security isn't a one-time implementation—it's an ongoing practice that requires constant vigilance and updates. The techniques and tools discussed in this guide will help you build more secure applications, but staying informed about new attack vectors and defense strategies is equally important.

The goal isn't to become a security expert overnight, but to develop a security-conscious mindset that naturally considers these risks as you design and build frontend applications. With practice and attention to these principles, you'll be able to create applications that users can trust with their data and their safety.

The framework I use is called RADIO, and it's basically a roadmap for tackling any frontend system design question without getting lost in the weeds. Let me walk you through it.

Why You Need a Framework

Here's the thing about system design interviews – they're intentionally vague. When someone says "design Instagram," they don't want you to build the entire app. They want to see how you think through complex problems, break them down, and communicate your thought process.

Without a framework, it's easy to jump around between topics, miss important parts, or spend 20 minutes talking about something the interviewer doesn't care about. Trust me, I've been there.

RADIO gives you structure: Requirements, Architecture, Data model, Interface definition, Optimizations. Each step builds on the previous one, and you'll naturally cover everything that matters.

R - Requirements: Figure Out What You're Actually Building

The first step is understanding what you're supposed to design. This might sound obvious, but "design Instagram" could mean a lot of things. Are we talking about the photo-sharing feed? Stories? Direct messaging? The camera interface?

You need to dig deeper and ask questions like you're working with a product manager on a new feature. Here are the types of questions I always ask:

What's the core functionality? For Instagram, this might be viewing photos in a feed, posting new photos, or browsing user profiles. The interviewer usually has a specific area in mind.

What features should I focus on? Even within the Instagram feed, there are tons of smaller features – liking posts, commenting, saving posts, sharing to stories. Figure out which ones are must-haves versus nice-to-haves.

What platforms are we targeting? Mobile web, desktop, native apps? This affects your architecture decisions later.

Any performance requirements? Should the feed load instantly? How many posts should we show initially?

Don't spend forever on this – maybe 10-15% of your interview time. But getting alignment here prevents you from going down the wrong path later.

Let's say we're designing Instagram's photo feed. After asking questions, we've agreed to focus on:

• Viewing an infinite scroll feed of photos

• Liking and commenting on posts

• Creating new photo posts

• Mobile-first design with good performance

Perfect. Now we know what we're building.

A - Architecture: The Big Picture Components

Now comes the fun part – designing the actual system. Think of this as drawing boxes and arrows to show how different parts of your app work together.

For frontend system design, I usually think about these types of components:

The Server – This is your API. Treat it as a black box that gives you data when you ask for it.

Views/UI Components – These are what users see and interact with. For Instagram, you'd have a FeedView, PostView, CommentSection, etc.

Controllers or State Managers – These handle business logic and coordinate between your UI and data. Think Redux, Zustand, or even just React hooks.

Client Store – Where you keep data that multiple parts of your app need. User info, cached posts, app settings.

Here's how I'd break down Instagram's feed:

• The Server handles all the backend stuff – storing posts, user data, handling likes. It exposes APIs we can call.

• The Feed Controller manages the feed logic – fetching posts, handling infinite scroll, coordinating updates when someone likes a post.

• The Client Store keeps track of cached posts, user session data, and app state that persists across different views.

• The Feed UI renders the actual interface – the list of posts, pull-to-refresh, loading states.

• Individual Post Components render each photo post with all its interactions – likes, comments, share button.

• The Post Creator handles the flow for adding new posts – photo upload, caption editing, posting.

Draw this out with boxes and arrows. It doesn't need to be pretty, but it should show how data flows through your system. When a user likes a post, the flow might be: Post Component → Feed Controller → Server API → Update Client Store → Re-render UI.

D - Data Model: What Information Lives Where

Once you know your components, you need to think about what data each one needs. There are basically two types of data in frontend apps:

Server data – Stuff that comes from your backend, like user profiles, posts, comments. This usually gets cached on the client.

Client-only data – Temporary stuff that only matters in the browser, like which modal is open, form input values, loading states.

For Instagram's feed, here's what I'd map out:

• Post data (from server): Each post needs an ID, the photo URL, caption, author info, like count, comments, timestamp. This lives in your client store after fetching from the server.

• User data (from server): User ID, username, profile picture, follower count. Also cached in your store.

• Feed state (client-only): Current scroll position, which posts are visible, loading state for infinite scroll, any error states.

• New post data (client input): When creating a post, you temporarily store the photo file, caption text, selected filters until the user hits "share."

• UI state (client-only): Which comments section is expanded, modal visibility, navigation state.

The key is understanding what data needs to persist (like cached posts) versus what can disappear when you refresh the page (like which comment section is open).

I - Interface Definition: How Everything Talks

Now you need to define how your components communicate with each other and with the server. This is basically your API design.

Server APIs are HTTP requests. For Instagram's feed, you'd need:

• A GET /feed endpoint that returns posts for the user's timeline. It takes pagination parameters (maybe a cursor for infinite scroll) and returns a list of posts with all their data.

• A POST /posts endpoint for creating new posts. Send the photo file, caption, and any metadata.

• A POST /posts/{id}/like endpoint for liking posts.

Client APIs are just function calls between your components. Like:

feedController.loadMorePosts() - Called when user scrolls near the bottom feedController.likePost(postId) - Called when user taps the heart postCreator.sharePost(photoData, caption) - Called when user finishes creating a post

The important thing is being clear about what data goes in and what comes back out. Your feed API might return something like:

{
  "posts": [
    {
      "id": "abc123",
      "author": {
        "username": "johndoe",
        "profilePic": "https://..."
      },
      "photo": "https://...",
      "caption": "Great sunset today!",
      "likes": 127,
      "createdAt": "2024-03-15T10:30:00Z"
    }
  ],
  "nextCursor": "xyz789",
  "hasMore": true
}

O - Optimizations: Making It Actually Work Well

This is where you get to show off your frontend chops. You've got the basic system designed, now how do you make it fast, accessible, and bulletproof?

Performance optimization is huge for something like Instagram's feed:

• Virtual scrolling for handling thousands of posts without crashing the browser. Only render posts that are actually visible.

• Image lazy loading so you're not downloading photos that users might never see.

• Aggressive caching – once you've loaded a post, keep it in memory so switching between tabs is instant.

• Optimistic updates for likes – update the UI immediately when someone taps the heart, then sync with the server in the background.

• Code splitting so your initial bundle is small and loads fast.

User Experience improvements:

• Skeleton screens while posts are loading instead of blank white screens.

• Infinite scroll that actually works smoothly – preload the next batch before users reach the bottom.

• Offline support using service workers and cached data.

• Good error handling when the network is flaky.

Accessibility considerations:

• Proper ARIA labels for screen readers.

• Keyboard navigation support.

• High contrast mode compatibility.

• Alt text for images.

You won't have time to cover everything, so pick 2-3 areas that are most relevant to the product or that you know well. If you're great at performance optimization, dive deep into that. If accessibility is your thing, talk about the challenges of making an image-heavy feed accessible.

Putting It All Together

The beauty of RADIO is that each step naturally flows into the next. Your requirements inform your architecture. Your architecture determines your data model. Your data model shapes your APIs. And your APIs give you concrete things to optimize.

Most importantly, you're having a conversation throughout this process. You're not just talking at the whiteboard – you're collaborating with your interviewer to build something together.

I've used this framework for designing everything from social media feeds to collaborative editors to e-commerce checkout flows. The specific details change, but the process stays the same.

Final Tips

Don't try to be perfect. System design interviews aren't about finding the one correct answer. They're about demonstrating how you think through complex problems.

Be opinionated. If you think infinite scroll is better than pagination for Instagram, say so and explain why. Interviewers want to see your decision-making process.

Ask questions throughout. If you're not sure whether to focus on mobile or desktop, ask. If you're wondering about scale requirements, ask. The interviewer wants to help you succeed.

Practice drawing. Whether it's on a whiteboard, tablet, or screen share, you'll be drawing diagrams. Get comfortable with basic boxes and arrows.

The next time someone asks you to "design Instagram," you won't panic. You'll start with requirements, sketch out your architecture, define your data model, specify your APIs, and discuss optimizations. Before you know it, you'll have designed a complete system that actually makes sense.

And honestly? That's a pretty good feeling.

You’re given an array of coin denominations (e.g. [1, 2, 5])
and a target amount (e.g. 11).
Return the minimum number of coins needed to make up that amount.
If it's not possible, return -1.

🎯 What You’re Really Being Asked

Find the shortest path to a number, using only steps defined by the coins.

For example:

coins = [1, 2, 5]
amount = 11

You could make 11 in a ton of ways:

• 1+1+1+...+1 (11 coins)

• 5+5+1 (3 coins)

• 2+2+2+5 (4 coins)

But you want the way that uses the fewest total coins.

💣 Why This Problem Is Confusing

Most people throw this at you:

minCoins[i] = Math.min(minCoins[i], 1 + minCoins[i - coin])

And you’re left going:

• Where is this coming from?

• Why is minCoins[i] being compared to itself?

• What the hell is this “bottom-up” thing and why does it work?

This blog answers all of that.

✅ Step-by-Step Plan That Actually Makes Sense

🧱 Step 1: Define the Meaning of DP

We create:

minCoins[i] = the minimum number of coins needed to make amount i

We’re going to fill this from 0 up to amount.

🚨 Base Case:

minCoins[0] = 0

Why?

Because it takes 0 coins to make amount 0.
(You don’t need anything.)

🧱 Initial Setup:

We assume all other amounts are impossible by default.
So we set them to “infinity” (amount + 1 is safe):

let minCoins = new Array(amount + 1).fill(amount + 1);
minCoins[0] = 0;

This says:

“I don’t yet know how to make amount 1, 2, ..., amount —
but I’ll try to find better answers as I go.”

🧠 Step 2: Core Logic — Build Up the Answer

Now the key idea:

for (let i = 1; i <= amount; i++) {
  for (let coin of coins) {
    if (i - coin >= 0) {
      minCoins[i] = Math.min(minCoins[i], 1 + minCoins[i - coin]);
    }
  }
}

Let's break it line by line:

🔁 Outer Loop: for (i = 1 to amount)

“Let me solve for amount = 1, 2, 3, ..., amount one by one.”

We’re building answers from the bottom up.

🔁 Inner Loop: for (coin of coins)

“For this current amount, let me try every coin I have.
Each one could be the last coin used to make this amount.”

🔥 The Real Brain of the Algorithm:

minCoins[i] = Math.min(minCoins[i], 1 + minCoins[i - coin]);

🚨 Read this in real words:

“If I use coin c now, then I only need to figure out how to build amount - c.
That’s minCoins[i - coin].
So I take that answer, add 1 (for the current coin), and see if it's better than what I had before.”

📈 Example: coins = [1, 3, 4], amount = 6

Let’s walk through how we build minCoins[6].

Start with:

minCoins = [0, ∞, ∞, ∞, ∞, ∞, ∞]
             0   1   2   3   4   5   6

i = 1

• Try 1: 1 + minCoins[0] = 1 + 0 = 1 ✅

• Others invalid

minCoins = [0, 1, ∞, ∞, ∞, ∞, ∞]

i = 2

• Try 1: 1 + minCoins[1] = 1 + 1 = 2

minCoins = [0, 1, 2, ∞, ∞, ∞, ∞]

i = 3

• Try 1: 1 + minCoins[2] = 1 + 2 = 3

• Try 3: 1 + minCoins[0] = 1 + 0 = 1 ✅

minCoins = [0, 1, 2, 1, ∞, ∞, ∞]

i = 4

• Try 1: 1 + minCoins[3] = 1 + 1 = 2

• Try 3: 1 + minCoins[1] = 1 + 1 = 2

• Try 4: 1 + minCoins[0] = 1 + 0 = 1 ✅

minCoins = [0, 1, 2, 1, 1, ∞, ∞]

i = 5

• Try 1: 1 + minCoins[4] = 1 + 1 = 2 ✅

• Try 3: 1 + minCoins[2] = 1 + 2 = 3

• Try 4: 1 + minCoins[1] = 1 + 1 = 2

minCoins = [0, 1, 2, 1, 1, 2, ∞]

i = 6

• Try 1: 1 + minCoins[5] = 1 + 2 = 3

• Try 3: 1 + minCoins[3] = 1 + 1 = 2 ✅

• Try 4: 1 + minCoins[2] = 1 + 2 = 3

minCoins = [0, 1, 2, 1, 1, 2, 2]

✅ Final Answer:

minCoins[6] = 2

You can form 6 using: 3 + 3 or 1 + 1 + 4 → both use 2 coins.

🔁 Forward vs Reverse Loop

You don’t need reverse loop in this problem.

Why?

Because every time you do:
minCoins[i] = 1 + minCoins[i - coin],
you’re only reading past values (from smaller i’s)

✅ Those values have already been finalized in earlier steps
✅ So forward loop works just fine

You’d use reverse loop only when:

• You must not reuse the same item in the same iteration

• Like in 0/1 Knapsack (where each item can be used at most once)

Not here. You can reuse coins as many times as you like.

🔚 Return Value

Finally:

return minCoins[amount] !== amount + 1 ? minCoins[amount] : -1;

If it’s still the "infinity" value → No solution found → return -1
Otherwise, return the best answer.

🧠 TL;DR Mental Model

• For each amount i

• Try every coin c

• Think:

“If I use c now, how many coins did it take to form i - c?”
Add 1 for the current coin
Keep the best result

That’s it. That’s the chain reaction you wanted to visualize.

💬 Final Thoughts

Coin Change is a classic — not because it's hard,
but because it's so easy to explain badly.

If someone had just said:

“Think of coins as steps, and amount as the top of a staircase.
You’re trying to reach the top with as few steps as possible,
trying every footstep that gets you closer,
and always remembering what the best path to each stair looked like.”

You’d have gotten it on Day 1..

All we do in these graph traversals, be it BFS or DFS, is follow a simple process:

1. We pick a cookie from the basket.

2. We find all its directly connected cookies (of different flavors like oats, almond).

3. We put these newly found, unvisited cookies into the basket.

4. We take a bite from the cookie we picked (meaning, we process it).

5. Then, we put this tasted cookie onto our plate.

We repeat this: pick a new cookie from the basket, find its connected cookies, add them to the basket, take a bite from the picked cookie, and add it to the plate which already holds the cookies we've tasted. We keep going until the basket is empty. At the end, we return the plate of tasted cookies, showing the order we visited them.

The only and only difference between BFS and DFS Traversals is in what order we pick the cookies from the basket. This difference in the order of picking cookies from the basket decides how the maze of connections will be traversed: either spreading out (breadth-first) or going deep (depth-first).

Now let's see how the code pulls it off.

First, let's imagine our cookie connections. It's like a map telling us which cookies are next to each other:

// Our yummy cookie connections!

// This is like our graph's adjacency list.

const cookieConnections = {

  'Chocolate': ['Almond', 'Oats'],      // Chocolate cookie is connected to Almond and Oats

  'Almond': ['Chocolate', 'Coconut'], // Almond is connected to Chocolate and Coconut

  'Oats': ['Chocolate', 'Cinnamon'],  // Oats is connected to Chocolate and Cinnamon

  'Coconut': ['Almond'],              // Coconut is connected to Almond

  'Cinnamon': ['Oats']                // Cinnamon is connected to Oats

};


// We'll start our tasting adventure with the Chocolate cookie

const firstCookieToTaste = 'Chocolate';

The Code Setup (Before the Main `while` Loop)

Here we simply set up our basket, the plate, and get ready to taste. We also need a way to remember which cookies we've already seen or put in the basket, so we don't process them again. Let's call this cookiesWeKnow. We then put the first chocolate cookie into the basket and mark it as known.

This setup is almost identical for both BFS and DFS:

function startTasting(connections, startCookie, isBFS) { // Added a parameter to show which function it is for clarity

    let basket = []; // Our basket for cookies to try

    let plate = [];  // Our plate for cookies we've already tasted

    let cookiesWeKnow = new Set(); // To remember cookies we've identified (either in basket or on plate)


    // Put the first chocolate cookie in the basket to kick things off

    if (connections[startCookie]) { // Make sure the starting cookie exists

        basket.push(startCookie);

        cookiesWeKnow.add(startCookie); // Mark it as known

    } else {

        console.log("Starting cookie not found in connections!");

        return plate;

    }

    // ... rest of the traversal logic will go here ...

The Main Tasting Cycle (The While Loop)

This while loop represents the entire cycle of picking a cookie, finding its connected cookies, adding them to the basket, tasting the picked cookie, and putting it on the plate, just like we discussed earlier. It will keep going as long as there are cookies left in the basket.

const currentCookie represents the cookie we have just picked from the basket.

Now here's the first big difference in BFS and DFS code that changes how we traverse the graph: how we pick currentCookie using JavaScript array methods .pop() and .shift().

• Stack Data Structure (DFS) using an array: .pop() takes the last element from the basket array (Last-In, First-Out or LIFO). Think of it as taking a cookie from the top of a pile in the basket.

• Queue Data Structure (BFS) using an array: .shift() takes the first element from the basket array (First-In, First-Out or FIFO). Think of it as taking the cookie that has been waiting the longest in a line in the basket.

In what order we put the cookies in the basket and, crucially, in what order we pick them up to taste, is what decides how the traversal will happen.

Let's see it in context for BFS:

// For BFS:

    while (basket.length > 0) {

        // Pick the cookie from the FRONT of the basket (FIFO)

        const currentCookie = basket.shift(); // Magic for BFS!


        // Take a bite and put it on our plate

        plate.push(currentCookie);


        // Now, let's look for its connected cookies...

        // ... (for loop for neighbors comes next) ...

And for DFS:

JavaScript

// For DFS:

    while (basket.length > 0) {

        // Pick the cookie from the TOP of the basket pile (LIFO)

        const currentCookie = basket.pop(); // Magic for DFS!


        // Take a bite and put it on our plate

        plate.push(currentCookie);


        // Now, let's look for its connected cookies...

        // ... (for loop for neighbors comes next) ...

Finding and Adding Connected Cookies (The For Loop)

Earlier we understood the order in which we pick a cookie from the basket is influenced by .pop() (for DFS) and .shift() (for BFS). Now let's focus on how the newly found connected cookies are added to the basket.

For BFS:

In BFS, after we pick a cookie (using .shift()) and put it on our plate, we find its connected cookies. We then run a straightforward ascending order for loop (from the first connected cookie to the last). If a connected cookie is new to us, we add it to the end of the basket (queue).

// Inside the BFS while loop, after plate.push(currentCookie):

        const connectedCookies = connections[currentCookie] || [];


        // In BFS, we run a straightforward ascending order for loop

        // to find new cookies and add them to the end of the basket (queue).

        for (let i = 0; i < connectedCookies.length; i++) {

            const nextCookie = connectedCookies[i];

            if (!cookiesWeKnow.has(nextCookie)) {

                cookiesWeKnow.add(nextCookie);

                basket.push(nextCookie); // Add to the end (FIFO style)

            }

        }

    } // End of BFS while loop

    return plate;

} // End of BFS tasting function

For DFS:

In DFS, after we pick a cookie (using .pop()) and put it on our plate, we also find its connected cookies. But here's a subtle and important detail: we typically run a reverse for loop (from the last connected cookie back to the first) when adding these cookies to the basket (stack).

Why? This is often done so that the traversal happens from "left-to-right" conceptually, if you imagine the connected cookies listed in a certain order (e.g., if 'Chocolate' is connected to ['Almond', 'Oats'], we want to explore 'Almond' and its path before 'Oats'). Since we use .pop() (Last-In, First-Out) for DFS, if we find neighbors ['Almond', 'Oats'] and add 'Oats' first, then 'Almond' to the basket, 'Almond' will be on top and be picked next. This achieves the "left-to-right" exploration. If we used a forward loop to add them, 'Oats' would be picked first, leading to an (equally valid, but different) "right-to-left" traversal relative to their order in the list.

// Inside the DFS while loop, after plate.push(currentCookie):

        const connectedCookies = connections[currentCookie] || [];


        // In DFS, we run a reverse for loop to add connected cookies to the basket (stack).

        // This ensures that if connections are [A, B], B is pushed, then A is pushed.

        // So, A (the "leftmost" or first in the original list) is on top and gets explored next.

        for (let i = connectedCookies.length - 1; i >= 0; i--) {

            const nextCookie = connectedCookies[i];

            if (!cookiesWeKnow.has(nextCookie)) {

                cookiesWeKnow.add(nextCookie);

                basket.push(nextCookie); // Add to the end, which is the top for .pop()

            }

        }

    } // End of DFS while loop

    return plate;

} // End of DFS tasting function

The Main Difference in a Nutshell

• In simple words, the key difference is that in DFS (Depth-First Search), we keep "piling up" newly found connected cookies, and the next cookie we taste is often one that was very recently added to the top of this pile (because we .pop() the newest). This makes us go deeper and deeper along one path of connections.

• On the other hand, BFS (Breadth-First Search) goes in a more "orderly" fashion. Cookies are picked one-by-one from the beginning of the line in the basket (because we .shift() the oldest). This leads to an equal traversal spread, exploring all cookies at the current "layer" or distance before moving to the next layer.

And that's how the code, with just a small change in picking from the basket (and how we strategically add to it for DFS), pulls off two very different ways of exploring our delicious world of cookies!

Let's see the order of tasted cookies for our example:

// To make the functions above runnable, let's wrap them:


// BFS Function

function bfsTraversal(connections, startCookie) {

    let basket = [];

    let plate = [];

    let cookiesWeKnow = new Set();


    if (!connections[startCookie]) return plate;


    basket.push(startCookie);

    cookiesWeKnow.add(startCookie);


    while (basket.length > 0) {

        const currentCookie = basket.shift();

        plate.push(currentCookie);

        const connectedCookies = connections[currentCookie] || [];

        for (let i = 0; i < connectedCookies.length; i++) {

            const nextCookie = connectedCookies[i];

            if (!cookiesWeKnow.has(nextCookie)) {

                cookiesWeKnow.add(nextCookie);

                basket.push(nextCookie);

            }

        }

    }

    return plate;

}


// DFS Function

function dfsTraversal(connections, startCookie) {

    let basket = [];

    let plate = [];

    let cookiesWeKnow = new Set();


    if (!connections[startCookie]) return plate;


    basket.push(startCookie);

    cookiesWeKnow.add(startCookie);


    while (basket.length > 0) {

        const currentCookie = basket.pop();

        plate.push(currentCookie);

        const connectedCookies = connections[currentCookie] || [];

        for (let i = connectedCookies.length - 1; i >= 0; i--) {

            const nextCookie = connectedCookies[i];

            if (!cookiesWeKnow.has(nextCookie)) {

                cookiesWeKnow.add(nextCookie);

                basket.push(nextCookie);

            }

        }

    }

    return plate;

}


console.log("BFS Tasted Order:", bfsTraversal(cookieConnections, firstCookieToTaste).join(', '));

// Expected BFS Output: Chocolate, Almond, Oats, Coconut, Cinnamon


console.log("DFS Tasted Order:", dfsTraversal(cookieConnections, firstCookieToTaste).join(', '));

// Expected DFS Output: Chocolate, Almond, Coconut, Oats, Cinnamon

// (This DFS order explores Chocolate -> Almond -> Coconut fully, then backtracks to Chocolate's other child Oats -> Cinnamon)

Checkout these youtube shorts where a Queue or Stack is the basket, and Processed is the tasted cookies Plate.

For BFS: https://www.youtube.com/shorts/umHJzlKFGlU

For DFS: https://www.youtube.com/shorts/5Kl3L3kdGbs

Next.js, a robust React framework, utilizes Server-Side Rendering (SSR) and client-side hydration to optimize how users perceive load times—exemplifying one such effective technique.

Understanding the Perceived Speed

When you visit a website and the content appears immediately, it gives the impression of a fast-loading page, even if the data fetching time remains constant (compared to web apps built using other languages and frameworks). Next.js uses this phenomenon by providing pre-rendered HTML content for the initial view, which swiftly gives users a sense of responsiveness. This means that users can see the content on their screen without delay, making the experience more seamless and enjoyable.

The Next.js Approach

Next.js, built on React, combines the best of server-side rendering and client-side interactivity. Here's how it achieves the illusion of speed:

Server-Side Rendering (SSR):

Next.js employs SSR to pre-render the page on the server based on the requested URL. This pre-rendered HTML content is sent as the initial response to the user's request on the browser.

Client-Side Hydration:

Simultaneously, Next.js transmits the required JavaScript bundle for the page. As this JavaScript loads and runs on the client side, React takes over and "hydrates" the initial HTML. This process adds interactivity, transforming the static content into a fully functional and interactive React application.

By separating the rendering of the initial content on the server and the application logic on the client side, Next.js balances perceived speed and interactivity.

The Illusion at Play

The trick here is all about how users perceive it. The first HTML content pops up fast, giving the illusion of a lightning-fast page load. The subsequent hydration process amplifies this impression, making the page interactive and fully functional. Users are captivated right from the start, staying immersed in the experience.

Benefits Beyond Perception

While this perceived speed greatly enhances user satisfaction, it also has tangible benefits. Search engines can efficiently crawl and index the pre-rendered content, improving the application's search engine optimization (SEO). Additionally, users on slower devices or connections perceive a faster experience due to the swift display of the initial content.

Conclusion

Next.js optimizes user experience by swiftly displaying an initial application view, giving a sense of speed. Using SSR with client-side hydration, Next.js efficiently prioritizes content delivery and interactivity. Striking the right balance keeps users engaged and satisfied. In today's fast-paced digital era, it's all about creating experiences that perform well and feel lightning-fast for the end-users.

Resource Utilization by Processes

Earlier, while talking about processes, we came across a command known as ‘top’. top is similar to ‘ps aux’ but rather than giving us a snapshot of the resource utilization by the processes, it shows us a live version where it keeps updating the details after every 3-5 seconds.

Earlier, we only understood the columns and the data in the rows but left the part above it. Let us go through the key-value pairs.

top – 14:21:58 tells us the current time. up 4:40 means the system has been running for 4 hours 40 minutes. 1 user means only one user is currently logged in the system. load average tells how much load the system is taking in form of averages (We will discuss this later in more detail).

The Tasks section tells us that a total of 263 tasks exist. Out of which 1 is running, 262 are sleeping, 0 stopped, and 0 are zombie.
In the %Cpu(s) section, 0.9us means the percentage of CPU time used by the processes that are not nice (Understand the concept of ‘Nice’ here). 0.8 sy is for the processes run by the system or Kernel itself. 0.0 ni is for the processes that are nice or have nice value. 97.9 id is the percentage of CPU Time that is idle.

Our CPU/Kernel is always looking for the input/output processes in the system executed by the user. CPU allocates a separate waiting time to catch and process these input requests. So, 0.4 wa is the percentage of time currently invested by the CPU in waiting for input/output streams.

It may happen that sometimes we interrupt a software or a hardware device interrupts some process. The CPU has to listen to these as well. 0.0 hi, 0.0 si tells us the amount of CPU time going into Hardware interrupts and Software interrupts.

0.0 st tells us the amount of time something is stealing out of the Operating System. It can be some other OS like Windows on the same machine if you have installed one.
MiB mem is for memory utilization, and MiB swap is for Swap Memory utilization. The value numbers are in megabytes.

Checking Resource Utilization by a Single Process

Maybe, you want to monitor just one process. It can be because the process is slowing down your system,

There may come a situation where a process is slowing down your system, like Google Chrome, after keeping too many tabs open. And you want to monitor that particular process that we suspect might be slowing down our system.

Run the command top –p [PID] where –p stands for PID. You can find the PID of a running process from the first column of the top command data.

Here you can see the result of the top –p 1 command. It shows us the details of the mother process.

List of Files (lsof)

Let us imagine a scenario where you plug in the USB in your computer and open a code file directly from the USB via VSCode. Or maybe, you opened a movie via VLC Media Player directly from the USB. Now, after closing everything that you opened directly from the pen drive, if you try to right-click and ‘eject’ the USB, it sometimes may say something like, the device is currently in use. You cannot eject it. That too after closing VSCode and VLC Media Player.

It means there is a file inside the pen drive folder that might be getting used by a process in the system.

Everything is a Process. In this context, even the VSCode and VLC Media Player are Processes.

There is a command lsof to know what files are currently in use by a Process. ls means list, o means of, and f means files. It gives us a list of all the files currently in use with the associated process.

It will help you find out why you cannot eject a particular device by letting you know what process is utilizing a file of your device. Now you can kill that process with the kill command using its PID.

Multiple Threads of Execution

Let us assume you are playing a car-racing multiplayer game. There are two cars. You are controlling car 1 and the other player, car 2. Wherever you two are sitting in the world, your movements will get reflected in each other’s screen. If the other player moves his car left/right, you will see it on your screen.

Two things are happening in your system while you play the multiplayer game on your computer:

• The position of the player is updated.
• The position of the player is sent to the internet.

It proves that the Kernel provides multiple ways of execution in the same program. Let us take one more example to understand this better.

Let us assume you are writing code in some editor, and it is automatically getting saved. So, two things are happening here.

• Code Editing
• Auto-saving

As we know, all software are processes. Let us understand this above concept in those terms. Now the process has one piece of code that is doing one thing. It can be updating your position in the game based on keyboard input or editing the code written by you. The other piece of code in the process is doing some other thing. It can be sending your changed position to the server so the other player can receive it or auto-saving your written code in the code editor.

These two ‘things’ that the process is executing parallelly are called ‘multiple threads of execution’ and are also known as ‘Lightweight Processes’.

There is a way to check all the processes running along with their threads by running the command ps m.

The threads with - in PID and TTY columns are threads. As the bash and ps m processes only have one thread of execution, there is only one line of - or thread under them.

Load Averages

Run the command uptime

It is the same as the first line of the top command output. Earlier, we left Load Averages for later. Let us understand them now.

Load Average is a good way of observing the load on the CPU. These three numbers mean how much the load average is on your CPU at the interval of 1 minute, 5 minutes, and 15 minutes.

What does the CPU load mean? It means the average number of processes waiting to be executed by the CPU.

Let us say you have a 1-core CPU. Assume it is like a single-lane road. It is busy during rush hour, and the lane is running at full capacity, so the load average is 1. If the traffic gets double what the road can handle at once, the load average will be 2. If the lane can handle 10 cars at once, but only five cars are there, the load average will be 0.5.

If the CPU load average is 1, it does not mean the CPU is slow. Because all CPUs today do not come with a single-core CPU (Understand the concept of cores and CPU Time in detail here).

If your CPU has 4 cores (quad-core), it is like having 4 road lanes. And now, if the CPU load is 1, then it means the CPU is only using 25% of its capacity.

vmstat command

By running the command vmstat, you can see a detailed version of how much memory your system is utilizing right now.

Let us go through the output.

procs section

• r - processes at runtime.

• b - processes at uninterruptible sleep.

memory section

• swpd - how much swap memory we have.

• free - free memory we have

• buff - memory for buffer

• cache - memory for caching

swap section

• si - memory swapped in the disk

• so - memory swapped out of the disk

io section

• bi - blocks received from a block device

• bo - blocks sent out to a block device

system section

• in - number of interrupts per second

• cs - how many context switches are there, which means how many processes it is giving time to per second

cpu section

• us - time, given to the user

• sy - time, given to the system

• id - idle time

• wa - waiting time for input/output streams

• st - steal time

Logging

Various types of processes are running in your system, like Kernel processes, daemon processes, and mother process etc. All these processes are continuously doing something in our system. And all these events are saved in a human-readable format in files known as ‘logs’. This activity of process events getting saved in files is known as ‘logging’. The logging data gets saved in the ‘/var’ directory.

How are logs getting created?

This is done through a service known as ‘Syslog’. It collects all the information on whatever is happening in our system and sends it to the ‘system logger’. One of the most important components of the Syslog is a daemon process known as ‘syslogd’. This Syslog daemon waits for any event to happen and then tries to figure out what it needs to do with it. It might be possible that it needs to send the event information to a file or print that message into the console, or maybe it does not have to do anything like in the case of /dev/null (this file nullifies whatever output we send to it).

System logger stores all the log files inside /var/log/syslog. But this is not the only file maintained by the System logger. There are more, like ‘/etc/rsyslog.d’.

As you can see, there are files for kernel logs, mail logs, mail errors etc. And in the second file, it is maintaining /var/log/ufw.log.

If you run ls /var/log, you can see the list of various maintained log files.

Logging Manual Messages

You can also send messages manually into your logs. Sometimes we may have done something we want to document in the logs. Or save some keywords inside so we know after which keyword we want to start searching in the logs. We have a command for this known as logger. If you run logger –s Hey my name is Gaurang Gaur

As you can see in the last line of the /var/log/syslog output, this message got stored as one of the log entries.

Log Management with logrotate

Logging also has to manage the disk space efficiently as the log files are taking so much data. And most of the time, we want to check the latest logs instead of the 3-month-old ones. All this management can be done through the ‘logrotate’ utility. It also has a config file where we can specify and configure things, like what logs we want to keep and how we want to compress our logs. The logrotate utility runs every day through ‘cron’ or cron jobs. You can find logrotate in /etc/logrotate.d, where there is a list of various configs like bootlogs, syslogs, daemon processes etc.

Thanks for reading :)

To read the previous blogs of this Linux Series, go to blog.gauranggaur.com

To see the video lecture of this blog, go to this youtube video

Follow me on Twitter

Do comment your thoughts, questions or anything unique you learned below!

When you start your computer, the booting of a Linux System goes through multiple stages.

BIOS (Basic Input/Output System)

BIOS initializes the hardware and ensures it is ready by running some self-tests. It checks if the memory, hard disk, and all the connected devices are good, whether the keyboard is connected, and many other similar checks. After this, BIOS does its main job of running the bootloader.

Bootloader

The Bootloader loads the Kernel into the memory based on the Kernel parameters. In the previous blogs, we understood that different config files help configure applications and processes existing in our system. Similarly, the Kernel has some parameters based on which the bootloader would initialize its code in the memory.
The Grub Bootloader is the universal Linux standard and is also very efficient if you want to dual boot your system with some other OS like Windows.

Kernel

Once the bootloader loads the Kernel into the memory, the third stage is the Kernel itself. The Kernel immediately initializes the memory and the devices present in the system. Then comes the main job of the Kernel, loading the ‘init’ or Mother Process.

init (Mother Process)

The mother process ‘init’ starts and stops all the essential processes that run our system.

Three Levels of Abstraction in Linux OS

A Linux Operating System can be classified into three levels of abstraction.

Hardware

It includes CPU, memory, ports, HDD/SSD, etc. Hardware is the physical layer that computes what we do on our system.

Kernel

In the previous Linux Masterclass Series blogs, we understood that the Kernel is responsible for handling the processes, the memory management between the processes, device communication, setting up the filesystem and system calls, etc.

User Space

All screen-related things we run, like the shell, the graphics we see, the chrome browser, etc., come under the User Space level of the Operating System.

Privilege Levels

The three abstraction levels we understood above also have three different privilege levels. A Kernel operates in a Kernel mode, and the User operates in User mode.

In the Kernel mode, the Kernel has complete access to the hardware and controls everything in the system. On the other hand, a user mode allows access to only a small amount of safe memory and the CPU.

Whenever we do anything related to the hardware, like reading/writing data and controlling network ports, we do it in the Kernel mode. And the reason behind that is simple. Imagine a scenario where spyware has infected your system. In such a situation, you wouldn’t want to access your hardware directly as the spyware can access your data, webcam and many other sensitive things. The two modes we talked about are also called Protection rings. Let us understand this in greater detail.

Understanding how Privilege levels work

Imagine there is a celebrity who is partying in a club surrounded by multiple layers of the crowd. The first layer is of the friends & family that he trusts the most. The second is of their bodyguards, the third is of the club bouncers, and the last fifth layer is of the general club crowd.

You exist in this fifth layer, and you want to get an autograph of the celebrity or a selfie with him. To achieve this, you might need a VVIP pass, Special Permission or something like that.

This is precisely how the protection rings/layers around the hardware work. The hardware is the celebrity. The layer/ring it trusts the most is the Kernel (friends & family). It is the ring ‘0’. And the last layer is ring 3 of the ‘user’. If the user wants to access the hardware and its resources, the VVIP pass here is a ‘System Call’.

System Call

System Call allows us to perform privileged instructions in the ‘Kernel mode’. The VVIP pass helps us get a similar treatment as a friend as we can talk to the celebrity, get a photo clicked etc. In the same way, a system call helps us perform some privileged instructions in the Kernel mode and then you can switch back to the User mode.

System calls are also known as Syscalls. The Syscalls provide the user-space applications/processes with a way for Kernel to do something for us. They make certain services available to users through System Call API.

All the System calls, like the weight system call where the Parent process wants an update on the child process termination status (Read more about it here) are just APIs that allow us to request Kernel to do something on behalf of us.

The Kernel offers us a fixed number of services through these APIs, and we cannot add more. Our System has a table of system calls where each has a unique ID.

Three Implementations of init, the mother process

System V

System V is the traditional implementation of the init process. It starts and stops processes sequentially. One of the advantages of using this implementation is that it is easier to resolve dependency issues.

Let us assume we have two processes: Process A and B, where B needs A to run itself. If Process A is not running, B cannot run. In such scenarios System V comes in handy as it ensures that the processes run sequentially. It will start Process A, and only then will start Process B.

But there is a disadvantage of System V implementation as well. Its performance is not good. The reason is that the processes start and stop one at a time instead of parallelly/simultaneously. That hinders the performance.

Runlevels

The status of System V is defined by ‘Runlevels’ that vary between 0 to 6.

• 0 -> Shutdown. If the system is shutting down, the runlevel will be 0.
• 1 -> Single User. If only a single user uses the machine, the runlevel will be 1.
• 2 -> Multiple users without networking. Multiple users are not connected through the network but locally.
• 3 -> Multiple users with networking.
• 4 -> Unused. When no one is using the system, and it is idle, the runlevel will be 4.
• 5 -> Multiple users with networking and GUI (Graphical User Interface). This is the state we are familiar with since we use GUI.
• 6 -> System is rebooting.

When the system starts up, it checks what runlevel you are currently in and executes all the scripts inside that particular runlevel config.

The runlevel config exists in the /etc directory. You can access it by running the command ls /etc/rc or if it says no such file or directory exists. Run ls /etc | grep rc. Here you are feeding the output of ls to the grep command and searching files starting with rc.

Now let us search the first one, rc0.d (runlevel 0 config) by running the command ls /etc/rc0.d

This is the list of all the scripts that will run at runlevel 0 or at the time of shutdown.

This is the list of all the config scripts that will run at runlevel 5, which we are familiar with. Today, the System V implementation is somewhat getting replaced. But these runlevels will still be there in other implementations, as they are fundamental to how the init works and not easily replaceable. The config files we saw above are only to support those services that start and stop through the System V scripts. System V has a lot of services, and there are many commands that you can use to manage them.

You can run the command service –status-all to check the status of all the services running through the System V init scripts.

To change their status, you can run the command sudo service [service name] [start/stop/restart]. The service name can be any from the list shown above. Write start to start the service, stop to stop it, and restart to restart it. This command system to change the status of a service is common among all init implementations. Since Linux is still in the middle of transitioning from System V implementation, this common command style is what is helping the transition.

Upstart

Upstart was used in the earlier implementations of Ubuntu, but today Ubuntu uses the Systemd implementation. Upstart was created to improve on the System V implementation.
It uses an events & jobs model that helps it to respond to an event as soon as it happens. Jobs are the actions it performs, and Events are the messages received from other processes that trigger the jobs.

To see a list of all the jobs along with their config, you can run ls /etc/init.

Now let us check one of these config files and run cat /etc/init/whoopsie.conf

As you can see, Whoopsie is a crash report submission daemon. If our system crashes, this daemon process reports it. start on runlevel [2345] means that the system needs to start the whoopsie daemon when the runlevels are either 2, 3, 4, or 5. And stop on runlevel [!2345] means that the system needs to stop the whoopsie daemon when the runlevels are not 2, 3, 4, or 5.

How does the Upstart Implementation work?

First, Upstart loads up all the configs from /etc/init. Then, when the startup event occurs, it will run all the jobs meant to be triggered by that event. The jobs that run through the startup event will trigger even more events. These events will run more jobs which might trigger even more events. So, this process keeps happening until all the jobs have run.

Systemd

Systemd is the latest, most emerging and adapted standard for init implementations. It uses goals to get your system up and running, where goals are the targets that you need to achieve. It is flexible and robust and does not do things sequentially to get the processes started.

How does the Systemd Implementation work?

It also loads up all the config files in the /etc/systemd and /lib/systemd/system directories.

Now let us go through the output of the second directory by running ls /lib/systemd/system | less. As the output will be quite long, we are using less with pipe (|). To understand less and pipe commands in deep detail with examples, click here

You can notice that there are a lot of files ending with .target. Let us understand the default.target file. Earlier, we understood in the Upstart implementation that the startup event triggered a lot of jobs, that in return triggered a lot of events, which again triggered a lot of jobs.

Now, in the Systemd init implementation, it first looks at all the configs and then all the goals it has when a system is booted. All those goals are inside the default.target file.
Run the command cat /lib/systemd/system/default.target for more information about this file.

It first tells us that this file is a part of systemd, followed by some information on systemd. Then, it tells us what this file requires, wants, conflicts, and what happens after. In short, it has everything a system needs to do on bootup. This file says it requires a multi-user.target file. Let us check that one out by running cat /lib/systemd/system/multi-user.target.

Now, this file says it requires a basic.target file. This is how the whole process happens on boot up led by default.target. They check each other’s dependencies, load each other and make our system work.

Systemd Targets

Similar to how System V has runlevels, Systemd has 'targets'.

• poweroff.target -> As the name suggests, it is for the shutdown.
• rescue.target -> It is for a single user.
• multi-user.target -> It is for multiple users with networking.
• graphical.target -> It is for multiple users with networking and GUI.
• reboot.target -> It is for reboot/restart.

The default.target is usually for graphical.target because GUI is the first thing we see. Managing Power States of the System

To shut down your system, you can run sudo shutdown –h now where the –h flag is equivalent to poweroff.

To reboot the system you can run the command sudo reboot. It simply changes the state of the system.

Since we are using the Systemd init implementation, these above two commands will be directed to the target file. And then, it will start completing those targets and their dependencies to complete the goal. That goal could be to shut down or reboot your system.

Thanks for reading :)

To read the previous blogs of this Linux Series, go to blog.gauranggaur.com

To see the video lecture of this blog, go to this youtube video

Follow me on Twitter

Do comment your thoughts, questions or anything unique you learned below!

When you connect any device to your system, it needs some device drivers to function. And we need device files to exist in the /dev directory for interaction with device drivers. You can view them by running the command ls /dev or, for a detailed view, run ls –l /dev.

You may notice that the standard output of this command is different from the usual ls –l output for any other regular directory like /home. Let us understand the unusual terminal output in the above screenshot.

Permissions

A 10-bit character set makes up a Permission set for a file/directory. We understand from the previous blog of this Linux Masterclass Series that ‘d’ represents a directory, ‘-’ represents a file, and ‘l’ represents a link. You can read that part of the blog along with the ‘rgx (read-write-execute)’ and ‘ugo (user-group-others)’ dynamic here in great detail

But here we can see more types like b, c, and p. ‘c’ represents a character, ‘b’ represents a block, ‘s’ represents a socket and ‘p’ a pipe (|).

c -> Character Device

This device transfers data but one character at a time. As we know, every piece of information is made up of bits. And a character device transfers one bit at a time.

b -> Block Device

This device transfers data in large fixed-size blocks. Devices like Hard Disk (HHD), and SSD are block devices.

p -> Pipe Device

Pipe or | allows two processes to communicate by taking the output of one command and transferring it as an input to another. Understand this in detail here

s -> Socket Device

This device also facilitates communication between processes like the pipe device but can communicate with multiple processes simultaneously.

Number of Hard links

It tells us how many hard links exist in this file. Discussed in much greater detail in the last section of this blog.

Owner

Group

Major Device Number, Minor Device Number

Some of the files have only one number, and some have two separated by a comma. This differentiation of major and minor is done for the ‘Device Characterization’.

The Major Device Number represents the in-use device driver of the device. For example, in the first line of the above output screenshot, autofs device is using the device driver 10. And in the third line, the btrfs-control device uses the same one.

Minor Device Number tells the Kernel, what unique device it is. The kernel uses this number to differentiate between different devices.

For example, in the above screenshot, the system has two nvme (one runs Windows, the other runs Linux), nvme0n1 and nvme1n1. They both have the same Major Device Number 259 as they use the same drivers. But as you notice, the Minor Device Numbers are different, like 0, 1, 2, 3, 4, 5, 6, and 7. The kernel uses these unique minor device numbers to identify each one. In case you are wondering what is p1, p2, p3, and p4, they are the partitions. nvme0n1 has 4, and nvme1n1 has 2.

###Timestamp

Device Name

Some devices in the above list are not connected physically to the system/motherboard. They are known as ‘Pseudo Devices’. They are /dev/null, /dev/zero, and /dev/random. Anything you enter/store in a ‘null’ device vanishes forever and never returns. It is like the black hole of your computer system. ‘random’ device is used to generate random numbers. All these three pseudo devices are character devices and transfer data one character at a time.

/sys Directory

The /sys directory gives detailed information about the /dev directory. Both are similar but have some major differences. While the /dev directory allows programs to access devices, the /sys directory helps to view information and manage devices. It stores device information like model, state, hierarchy, device manufacturer, where it is plugged into the system etc.

The numbers inside /sys/dev/block are the Major and Minor Device Numbers separated by a colon. The first one is Major the other is Minor. Their colour is light blue because they are link type. If you run ls –l /sys/dev/block/7:0, you can see more information, related to the devices using this driver.

Filesystem (fs) Hierarchy

If you run the command ls –l / where / represents the root directory. It will show a detailed list of all the root folder files. The whole tree of filesystem hierarchy in the Linux System starts from this root (/) directory.

This list also consists of /sys and /dev we understood above. Let us understand the other important directories and files in this root folder.

• bin directory -> It contains essential and ready-to-run binaries for commands like mkdir, ls, cd, and many more like mongod, mongoose etc.

• boot directory -> It contains the bootloader files used by the Kernel.

• dev directory -> It contains the device files

• etc directory -> It is the core system config directory. It contains all the configuration files of any program you run. Most programs you run have configurations/settings that you can influence to change their functioning.

• home directory -> it is the personal directory of the user. This is where your general-purpose directories are like Desktop, Documents, Downloads, Pictures etc. And the location where the terminal opens up by default.

• lib directory -> It contains the library files that the binaries might use. Binaries of the commands might need some external libraries to execute their code, like in web development.

• media directory -> It is an attachment point for any USB or something like that. If you connect a USB to the system, this is where it gets connected and accessed.

• mnt directory -> It is the temporarily mounted filesystem. The entire filesystem gets temporarily mounted here.

• opt directory -> It contains optional software packages. For example, if you are using Google Chrome, you will find its name inside it if you run the command ls /opt

• proc directory -> It contains the Process information. You can understand it in much greater detail here

• root directory -> It is like a home directory for the root user just like ‘/home’ is for a regular user.

• run directory -> It contains information about the running system since boot. From the moment you start your system, whatever information the system collects gets stored in this /run directory.

• sbin directory -> It contains the system binaries. The earlier binaries directory was for commands that any user could run. But the system binaries can only be executed by the root user.

• sys directory -> It helps in managing the filesystem.

• tmp directory -> It is the storage for temporary files.

• usr directory -> It contains the user-installed software and utility.

• var directory -> var stands for Variable. Any system login, user tracking, caches, and many other constantly changing things are variables maintained in this variable directory.

Filesystem (fs) Types

There are different implementations of filesystems. Some are faster than others, and some support larger storage. Different filesystems have different ways of structuring their data. The structure we see in the above section of the root directory is not the same for every system. It might be different based on the requirements.

Let us say we have an app that runs on a filesystem. The app doubts whether the filesystem supports the way it runs. In such a situation, we can take the help of an abstraction layer between the app and the filesystem known as a ‘Virtual File System’. If the app wants to operate on this filesystem, it will try to use the Virtual File System. Now, it is the job of the Virtual File System to ensure that the app file is operable on the filesystem, no matter the structure.

You can have multiple filesystems in your computer machine depending on how you have created the partition. We saw in the above section that there were two partitionsnvme0n1 and nvme1n1, one for Windows and the other for Linux.

Journaling

Let us say you have a large file and are copying it from one place to another. Since this is a large file, it will take a lot of time for copying to finish. Suppose the system battery gets depleted or your system shuts down for some reason, and the large file you tried to copy, gets corrupted. Because of this, there will be inconsistency in the filesystem. When you restart your system, it will perform some filesystem checks to make sure everything is okay.

You might have noticed when your computer unexpectedly shuts down takes a lot of time to boot back up on restart. Filesystem checks are one of the reasons why.

A process known as ‘Journaling’ helps you avoid such situations. It is part of the majority of operating systems that come these days. With Journaling, before your system begins to copy a file, it will write down what the user is attempting in a log file. This log file is known as a ‘Journal’. Once the task gets completed, like copying a file, it will be marked as ‘completed’ in the journal.

With the Journaling feature, your filesystem remains consistent. And if suddenly your system shuts down, it would know what it was trying to do before powering off by checking the journal. It results in the boot-up being much faster.

Desktop Filesystem Types

• ext4 –> This is one of the latest versions of Linux Filesystem and is compatible with previous filesystems like ext3, ext2 etc. It supports disk space of up to 1 exabyte and a file size of up to 16 terabytes. ext4 is the standard choice for a Linux Filesystem.

For Reference

8 bits -> 1 byte

1024 bytes -> 1 kb

1024 kb -> 1 Megabyte

1024 Mb -> 1 Gigabyte

1024 Gb -> 1 Terabyte

1024 Tb -> 1 Petabyte

1024 Pb -> 1 Exa byte

1024 Eb -> 1 Zeta byte

1024 Zb -> 1 Yota byte

1024 Yb -> 1 Bronto byte

1024 Bb -> 1 Geop byte

• Btrfs or Butter/Better File System -> This is also one of the latest Linux filesystems and comes with many things like snapshots, incremental backups, and performances. It is a new filesystem and yet not stable.

• XFS –> It is a high-performance journaling filesystem. It is generally good for servers that host media like YouTube and Netflix.

• NTFS & FAT –> It is used by the Windows Operating System.

• HFS+ -> It is used by the macOS.

You can check what all filesystems your computer system uses by running the command df –T, where the T flag displays a column with filesystem type.

Partition Table

We already know from the above sections that the HDD and SSD are block devices. And we can create partitions inside the block devices to separate data. Instead of having one filesystem in the entire HDD, SSD or nvme, you can create multiple partitions that act as individual block devices. This process will let you have more than one filesystem.

For example, in your 1 Tb SSD, you can have two 500 Gb partitions, one where Windows is running and the other for Linux. If we have some space that is not a part of any partition is declared as Free Space.

To check what partitions you have created, you can check the ‘Partition Table’. It also tells where the partition begins and ends, which are bootable, and which sector we have allocated to a partition etc.

There are two Partition Schemes:

1. MBR or Master Boot Record

2. GPT or GUID Partition Table

MBR

MBR is the traditional partition table that used to be the standard early on. It only supports disks up to 2 Tb. The problem with MBR is that it has a limit of four partitions known as ‘Primary Partitions’. But you can opt to have only one partition and make it an ‘Extended Partition’. In this Extended Partition, you can create as many ‘Logical Partitions’ as you want that behave as Primary partitions. But you can only choose one Primary partition to extend.

GPT

GPT is becoming the new standard for disk partitioning. Unlike MBR, GPT only has one partition type, and you can divide it as much as you want. Each partition will have a Globally Unique ID (GUID). GPT is used with UEFI-based Booting.

Filesystem Structure

A filesystem is an organized collection of files and directories. It’s like a database to manage the files and the actual files themselves. Any Filesystem Structure has four major components:

• Boot Block – It is in the first few sectors of the filesystem. It is only used to boot the Operating System.

• Super Block – It is the single block that comes after the boot block and generally contains the information about the filesystem itself. Information like the size of the Inode tables, filesystem etc.

• Inode Table – It is like a database that manages our files. Each file or directory has a unique entry in the Inode table that contains information about them.

• Data Blocks – These blocks contain the actual data of the files and directories.
  We can run the command sudo parted –l to view the partition detail of a system.

As you can see, Partition Table is ‘gbt’ which I explained earlier. Inside the table, the first one is for the Boot Block, which starts at 1024kb and ends at 538MB. The second part is the Data Block dedicated to the Linux OS with an ext4 filesystem.

There are different disk partitioning tools you can use:

• fdisk – It is a CLI-based tool, and it does not support GPT.

• parted – It is also a CLI-based tool. It supports both MBR and GPT.

• Gparted – This is the GUI-based version of parted.

• gdisk – It is also CLI-based and does not support MBR.

/etc/fstab File

If you want automatic mounting of a filesystem at system startup, you can add it inside this /etc/fstab file. You cannot read or go through a filesystem without mounting it to the system. For example, Linux is mounted in the system, and that is how I am able to use it on the machine. Earlier, we saw the ‘mnt’ directory, which is the place where this Linux OS is mounted.

If you run cat /etc/fstab, it will show a bunch of information. Let us go through some data points inside it.

It is showing UUID=4b6264c6-9bea-42a9-8f11-956f53700ea7. It is the unique identifier for the Linux Filesystem. In front of it, there is the directory / (root), where the Linux Filesystem is mounted. Then, there is the Filesystem type ext4. Then, we have the mount options, followed by dump and pass columns. We use the dump utility to create a backup. Pass system means in what order your filesystem should be checked, in case some discrepancy occurs.

The last line mentions /swapfile and swap. Let us understand them.

Swap Memory

We use swap to assign some virtual memory to your system. If you are low on memory or have a low amount of RAM, you can create a Swap Partition. The system uses this swap partition to swap memory pieces of idle processes to the disk.

The system takes out some pieces of memory that some idle processes occupy in the RAM and saves them into the swap partition inside the disk. This process will free up some memory in your system to be used by the active processes.

Disk Free and Disk Usage (‘df –h’ and ‘du –h’ Commands)

If you run the command df –h where the h flag shows a human-readable version of the output, it will show the currently mounted filesystems utilization. It includes their size, usage, availability, use percentage, and where we have mounted the filesystem in the machine. For example, the Linux filesystem is /dev/sda2 in the third row with a size of 916 Gb and mounted in the / (root) directory.

If you want to see what is taking how much space in your whole system, you can run the command du –h.

Inode Table

In the Filesystem Structure section, I explained that Inode Tables are like a database to manage files. In this table, each file/directory has an ‘Inode’ that stores all the information about it.

That information can be like file type, owner, group, permissions, number of hard links, size, number of blocks allocated, and the pointers to data blocks of the file. It is everything except the filename and the file itself. Because a file is stored on the disk and not in the inode table.

When we create a filesystem, some space for Inodes is allocated as well. There is an algorithm that determines how much inode space would be needed based on the size of the disk. You might be familiar with the error that says, ‘You are out of disk space’, but when you check your disk usage, you notice that there is still a good chunk of it left. It means your disk has space but the space allocated for your inode table does not have space anymore.

Data Storage on our disks depends on both, the data and the database (Inode Table). If you run the command df -i, it will show the total inodes available for each filesystem, how many you have used, how many are free, and the usage percentage.

Now, let us assume that you created the total number of empty files & directories equal to the available inodes for the Linux filesystem. Now even if the files are empty and your disk usage is not filled, you still would not be able to create/download any more files and directories in your system. Because there would be no space left in the inode table to add more entries for any new files/directories.

Inode Information

We identify Inodes by numbers. Whenever a new file/directory gets created, they are assigned an inode number. Inode numbers are usually in sequential order, but sometimes you may get a lower inode number. Because when you delete a file/directory, the associated inode number gets freed. And when you create a new file/directory, it may get assigned the same free inode number.

You can check the inode number of any file/directory by running the ls –li command. The first column consists of uniquely assigned inode numbers. To see a more detailed view of a directory, you can use the command stat [directory name or path].

If I run stat /, it will show me the information of my root directory. It consists of the size, Blocks, UID, GID, Access (permissions) in both numerical and graphical form, type (which is directory), Inode number, number of links, and some timestamps.

How does an Inode locate a file?

Let’s understand this with an example. You created two files: ‘First name’ and ‘Last name’. You created the variable ‘Elon’ in the ‘First Name’ file that gets stored in block 1 of main Data Storage. Then, you created the variable ‘Musk’ inside the ‘Last Name’ file that gets stored in block 2 of Data Storage. Then, you created the variable ‘Mark’ in the ‘First Name’ file that gets stored in block 3 of storage. Then, you created the variable ‘Cuban’ in the ‘Last Name’ file that gets stored in block 4 of storage.

Now, Blocks 1 & 3 belong to the ‘First Name’ file and Blocks 2 & 4 belong to the ‘Last Name’ file. You can notice that the files’ data is not stored sequentially. We know this data is stored somewhere in the Data Storage, but we don’t exactly know where it is as it was not stored sequentially.

In such a situation, Inodes help in finding the desired data. Each Inode contains 15 pointers. This set of 15-pointers is also known as a ‘Super Block’ which I explained above in the Filesystem Structure section. Each Inode pointer or block out of 15 has a size of 4 kb. The first 12 pointers point directly to the data blocks of the file. For the ‘First Name’ file, the pointers will point to the 1st & 3rd data block and for the ‘Last Name’ file, the pointers will point to the 2nd & 4th data blocks.

But if a single Inode pointer is 4 kb, does that mean a single Inode can only contain 48 kb of max data? The answer is no. This is when the 13th, 14th, and 15th blocks/pointers of an Inode come into the picture. They can point to a group of pointers (an Inode). And let’s say that Inode will again point to another Inode that at last points to a comparatively much larger number of data blocks compared to 15.

For example, Inode A’s 13, 14, and 15 Pointers point to three different Inodes. Those three Inodes’ 13, 14, and 15 pointers point further to three different Inodes. This is how a single initial Inode can point to a much larger number of data blocks.

Earlier I told you that an ext4 filesystem has 1 exabyte of disk space and 16 terabytes of file space. The vast network of Inodes we saw above fills this 16 TB of file space. In other words, this network of pointers pointing to other Inodes can go to a maximum size of 16 TB.
All the Inodes that come between the first initial Inode and the data are known as ‘Indirect Blocks’.

Links

Symlinks/Soft Links

If you have used Windows OS, you might be familiar with shortcuts. When we install a file, Windows asks us that do we want a shortcut for this file on the Desktop. These shortcuts are Aliases of the directory/file/program that we are shortcutting at a particular location (like, Desktop).

If you make some changes or delete the original file/directory, it will affect the shortcut as well and might break it. If the original file ceases to exist, where will the shortcut lead to?
In Linux, Symlinks/Symbolic Links/Soft Links are equal to shortcuts in Windows. To create a soft link of a file, you can run the command ln –s [filename] [SoftLink name]. Here ln stands for a link, and the -s flag stands for a soft link.

As you can see, there is one more file created. It is a soft link and is pointing towards the test1 file. And in the permissions column, instead of -, it has l, which means it is a link type. To understand Permissions in deep detail, go here

Hard Links

To create a hard link, you only have to remove the -s flag from the above command. Run a command like ln test2 hard2 to create a hard link.

You can notice that earlier, the number column after the Permission column consisted of only the digit ‘1’, but now test2 and hard2 have ‘2’ in their rows. Because this column tells us how many hard links of an Inode number exist in the system. Now that we have created a hard link of the ‘test2’ file, the number to ‘2’ from ‘1’.

Notice that the Inode number of hard2 and test2 is the same, contrary to soft1 and test1, which have unique Inode numbers. But earlier, we learned that each file/directory we create should have a unique Inode number. Let us understand what is happening here.

There is an Inode number and a file point to it. If we now create a soft link, it will point to the file instead of the Inode number. But if we create a Hard link, it points directly to the same Inode number the file is pointing to. If this file gets deleted, the soft link will also stop working. But since the hard link is pointing directly to the Inode, it will still work.

It does not mean that if you make any changes to the original file, they will not reflect in the hard link. They will. As the changes are made in the data storage. And both the file and hard link are pointing to the same data storage blocks via Inode.

If you make any changes to the soft link file, they will reflect in the original file as well.

If you make any changes in the hard link file, they will reflect in both the original file and the soft link file.

If you delete the original file, then the soft link file will not work, but the hard link file will work properly.

Thanks for reading :)

To read the previous blogs of this Linux Series, go to blog.gauranggaur.com

To see the video lecture of this blog, go to this youtube video

Follow me on Twitter

Do comment your thoughts, questions or anything unique you learned below!

The terminal shows us four columns.

• PID - Process ID

• TTY – 'Controlling Terminal' of the process. Most Processes we run are associated with a terminal. The ID of that terminal is written in the TTY.

• TIME – Total CPU Usage Time a Process is taking

• CMD – CMD (Command) is the name of the executable command/program currently running in the system.

If you open another terminal window while the first one is still open and run the ps command, the TTY value will change from 0 to 1.

ps aux command

In the ps aux command, a, u, and x are the flags of the ‘ps’ command.

• a – All the processes, even the ones run by other users
• u – More details about the processes
• x – All the processes, even the ones running in the background with no associated terminal (TTY).

If you run the ps aux command, it opens a large output with more rows and columns.

• USER – Every Process has an owner.
• PID – Process ID.
• %CPU – Percentage of CPU time a Process is taking.
• %MEM – Percentage of Memory a process takes out of the entire physical memory of the system.
• VSZ – When a Process starts up, Virtual Memory Usage or VSZ is the maximum memory in kb available to the entire Process. It can use some portion or all of the allocated memory but not more.
• RSS – The amount of memory in kb currently used by the process out of the total VSZ.
• TTY – Earlier, we saw the TTY column only consisted of values like ‘pts/o’ or ‘pts/1’, but now it also has ‘?’, ‘tty2’, and ‘tty3’ as the values. ‘?’ means no terminal associated. We’ll understand the other values below in the Controlling Terminal section.
• STAT – This tells us the process status.
• START – The exact time a process started.
• TIME – Total CPU Usage Time a process is taking.
• CMD – Name of the command/program.

top command

If you run the top command, this will show the real-time version of the ps aux data and much more. Here the values keep fluctuating based on what happens in the system. Press q to get back to the terminal.

The top command helps us to find out what processes might be slowing down or affecting the system in any other way.

Controlling Terminal (terminal and pseudo-terminal)

Earlier in the TTY section of ps aux, we saw two types of values.

Tty1, tty2 or tty3 – Regular Terminal

pts/0 or pts/1 - Pseudo Terminal

A regular Terminal is a terminal that has no graphics associated with it. It is only a shell where you can only use a keyboard to type commands without any GUI elements. Type ‘Ctrl+Alt+F3' to access TTY3, and to exit type ‘Alt+left arrow’.

The processes not associated with any terminal and run without it are known as Daemon Processes. They start when the system boots up and terminate when it shuts down. The daemon processes run in the background, ensuring our system runs fine. We do not want them to close, as our system might crash without them. That is the reason they are not associated with a controlling terminal.

All the processes in the ps aux output with ‘?’ under the TTY column are essential processes without any associated terminal. Because if we accidentally close a terminal that is running one of them, our system will crash.

Processes are an instance of a Program

Open three terminals and execute thecat command in two of them with or without an argument. And in the third one, execute ps aux | grep cat where you are transferring the output of ps aux to the grep command that will search the input for ‘cat’ and show output in the terminal with all the lines that contain ‘cat’.

Notice that there are two instances of cat, one running in pts/1 and the other in pts/2. Although both terminals execute the same command or program, we still have two instances. If you close the other two terminals, there will be no instances of cat running in pseudo-terminals anymore. This example proves that ‘processes’ are instances of a running program. We can have as many instances of a program in our system as we want.

Who manages the Processes? And how?

By managing, I mean who reserves/allocates and makes decisions about the VSZ memory, CPU time etc., for a process to use. The answer is the ‘Kernel’. The Kernel manages all the processes in the system. It loads up the code of a program in the memory, allocates all the resources that the process might need, and keeps all of them in check. It knows the status, the owner, and the resources allocated/consumed by a process.

When a process ends, the Kernel freezes up the resources the process was using and makes them available for other processes to use.

Now let us come to the How part. When the user creates an instance of a program, it clones using the ‘Fork System Call’. Fork System Call creates an identical child process (or we can say, forks it) and assigns a Process ID (PID) to it. The original process that this child process was cloned or forked from becomes the Parent Process.

Parent & Child Processes

A Child Process can use the same ‘Program’ that its parent process was using or in most cases, it can use ‘execve system call’ through which it can run a new program. If you run the command ps –l, it shows a more detailed view of the ps command output where there is a new column called PPID.

The ps command has a personal ‘PID’ of 13087 and a ‘PPID’ of 11530. This PPID of 11530 is the same as PID 11530 of the bash command. So, we can say the ‘ps’ command process is the child process of the ‘bash’ command. But you can see the ‘bash’ command also has a PPID, which means it also has a parent process. We can find it out by running ps aux | grep 11522 where 11522 is the PPID of bash.

We can see that bash is the child process of /usr/libexec/gnome-terminal-server.

Mother Process

Now, you may wonder if every process is the child of its parent process, then there must be a process above all other processes, the Mother process. This Mother Process is known as ‘init’. When the system boots up, it creates the mother process ‘init’ with the PID of 1. ‘init’ runs on root privileges with the ‘root’ user as the owner and is responsible for most of the processes responsible for running the system.

By running the ps aux command, the first line shows the ‘init’ process with the PID of 1 and owner as ‘root’. It has the highest ‘VSZ’ allocated and ‘RSS’ used. All the other processes below, with TTY as ‘?’ and PIDs consecutively increasing, are the Daemon Processes essential for running the system. They are all child processes or forks of the ‘init’ mother process.

Termination of a Process

The system terminates a Process using the ‘-exit system call’. It is responsible for freeing up all the utilized resources. When a process terminates, it lets the kernel know why it is terminating with a ‘Termination Status’. The most common Termination status is ‘0’, which means ‘the process succeeded’.

When a child process terminates, its parent process has to acknowledge its status with the ‘wait system call’ and issue the termination status.

There are different use cases when it comes to the termination of a process. Let us understand them.

Orphan Process

In this case, the parent process gets terminated instead of the children. These Child Processes are known as Orphan Processes. They do not have a Parent Process that can use the ‘wait system call’ to acknowledge the termination of their child. In such a case, the Kernel puts such Orphan Processes under the control of the ‘init’ mother process.

Zombie Process

In this case, the parent process exists but does not acknowledge the termination of the child process and does not call the ‘wait system call’ for some reason. So, the Kernel turns such a Child Process into a ‘Zombie Process’. It is living dead, meaning it is terminated, but still exists because the system does not know its termination status.

Let us assume after the child process turns into a zombie process, the Parent process calls the ‘wait system call’. This is known as ‘reaping’. But if the ‘reaping’ does not happen, the ‘init’ process adopts the zombie process, calls the ‘wait system call’ and issues a termination status so the zombie process can rest in peace.

Signal

A Signal is a notification sent to a Process that something has happened. When a user kills or suspends a process, the process needs to know that it has been killed/suspended only then the user action will transpire. Maybe some hardware or software issue occurred, and the Kernel wants to notify the Process. All this happens with the help of signals.

There are many things a Process can do with a signal it receives. It can ignore the signal or catch it and perform a signal handler routine based on the configuration. All processes have signal handlers that are responsible for handling different signals.

For example, the software is running and as soon as it receives a termination signal saying it has to stop working and terminate itself, the signal handler will save the collected data and terminate the process. There are different types of signals:

• SIGHUP/HUP/1 – Hang up
• SIGINT/INT/2 – Interrupt
• SIGKILL/KILL/9 – Kill. No Signal Handler of any process can block this signal.
• SIGSEGV/SEGV/11 – Segmentation Fault
• SIGTERM/TERM/15 – Terminate
• SIGSTOP/STOP/19 - Stop

If you run the command kill 15345 where 15345 is the PID, it will terminate the process. And the ‘exit system call’ will be called to free up all the occupied resources in the system. But let us say you do not want to terminate but kill the process entirely.

You can run the command kill –9 15345 where 9 is the numerical representation of the KILL signal. It will destroy the process instantaneously, but the occupied resources might not free up, which the Kernel might have to do later.

SIGHUP Signal is sent when the terminal on which the process was running is closed. In simple words, it tells the Process that you do not have any medium to work anymore as your terminal is closed, and you can hang up.

SIGINT Signal is sent to terminate/interrupt the Process when we press Ctrl+C after running the command.

SIGSEGV Signal occurs when the process tries to access some area of the memory it does not have access to. For example, We run the command cat /etc/shadow, but as we know that it is a root file and the user does not have access to it, this is when the segmentation fault occurs.

SIGTERM Signal kills the Process but after doing the cleanup like freeing the allocated memory etc., unlike SIGKILL.

SIGSTOP Signal only suspends a process for some time after which the process might resume.

CPU Time

We generally run multiple software in our system simultaneously like, Chrome, Terminals, VSCode etc. It may seem like all are running simultaneously, but that is not the case. All the processes use the CPU for a small amount of time, known as a ‘Time Slice’.

All the Processes use a short amount of CPU time (time slice in milliseconds) one by one in a cycle. Once a process gets its time slice, it usually pauses for a few milliseconds allowing other processes to use that freed CPU time slot.

This is where the number of cores in the CPU plays a role. A CPU can be dual-core (2), quad-core (4), octa-core (8) etc. The more cores a CPU has, the more Processes it can run simultaneously. For example, if your CPU has four cores and runs 100 Processes, each core in the system handles 25 processes.

The Kernel handles this switching of CPU time among the Processes. If a process gets this control, it will tend to keep a disproportionate amount of CPU time. We have a way to influence the amount of CPU time a process can have with a value known as ‘nice’. nice and renice commands

All the Processes running in the system have a number to determine their priority for the CPU for the amount of time slice the CPU should give them.

When you run the command top, there is a column ‘NI’ which contains values like 0, -20, 19, and 5. The higher the value, the nicer the process is.

High NI Value -> Low Priority

Low NI Value -> High Priority

If a process has a Low NI Value means it is not very nice. So, the CPU has to give more time to run that process smoothly and keep it on high priority. On the other hand, if a process has a High NI Value, the CPU does not have to prioritize it as the process is already very nice and will run smoothly even on a thin Time Slice.

You can change this NI Value with the command nice. For example, if there is a process that is not nice and you want the CPU to keep it on High Priority and a lot more of its time to run it faster, you can do that by running the command nice –n –50 [process name].

For the processes already running on the system, you can use the command renice like renice –10 –p [PID] where PID is of the already running process, and you want to change its NI value.

#States of a Process

• R State – Running or Runnable. It means the process is running or is waiting for the CPU to give it time to process.
• S State – Interruptible Sleep. It means the process is waiting for some event which can be input from the user.
• D State – Uninterruptible Sleep. It means the process cannot be killed or interrupted by a signal. The only way to do it is to reboot your system.
• Z State – Zombie State. It means the process is terminated and waiting for their ‘wait call status’ to be collected. We understood this in the above ‘Termination of a Process’ section.
• T State – Stopped State.

How everything in Linux is a file - the ‘/proc’ directory

In the first blog of this Linux Masterclass series, we talked about how everything, even the commands, processes, and directories, are all files in the Linux system.

‘/proc’ is the directory that contains all the information about all the processes in the system. If you run the command ls /proc, it will show a large list of the same.

All the subdirectories of this /proc directory in the output above are all the processes running in the system. Most of their names are their respective PIDs (Process IDs). These subdirectories contain all the information related to these specific processes.

If you run `ls /proc/1', the first subdirectory in the output which belongs to the ‘init’ mother process will show up.

Three of the results we are not permitted to see as they belong to the root. All these results are information about different processes in the form of files.

The /proc directory is from where the Kernel sees the system. The Kernel does not see everything the user is doing in a system at all times. The Kernel saves all the processes running in the system inside the /proc directory as files or subdirectories.

Whenever the Kernel wants to see what is happening in the system, what process does it need to give CPU Time Slice to, allocate/take resources or switch processes in the core or something else, is known by the Kernel with this /proc directory only.

Jobs

There can be a situation when you run a process in the terminal, and it takes a lot of time to execute, and until it finishes, you cannot run any other command in the same terminal. This is where the concept of ‘Jobs’ comes in. By writing & at the end of any command like python3 test.py &, it will run asynchronously and provide you with a Process ID.

As you can see, the terminal has provided the PID (21266) of this file and the Job ID ( [1] ) as the output running in the background. We can view all the jobs running in the background with the command jobs. It shows the exact command (process), the status as ‘Running’ and the Job ID as 1.

Now, if we write another command like cat &, it will show the Job ID as 2 with the + sign in front of it. The python command's Job ID sign has changed from + to – because it is older in chronological order. If you write another cat &, its Job ID will be 3 and the sign will be + while the previous command sign has changed to – and the python one has gone blank.

To bring a job from the background to the foreground, you can run the command fg %1 where 1 is the Job ID of the Process.

Let us say I run the command python3 test.py and sometime later decide it is taking a lot of time, so I should send it to the background. I can click Ctrl+Z to stop it first. Remember, Ctrl+C terminates a process, but Ctrl+Z stops it. If we now write the command bg, it will send it to the background and make it run once again.

If I want to kill this command midway, one way is to run fg %5 where 5 is the Job ID, bring it to the foreground and press Ctrl+C. The other way to kill the process without bringing it to the foreground is by running the command kill %5 where 5 is the Job ID.

Thanks for reading :)

To read the previous blogs of this Linux Series, go to blog.gauranggaur.com

To see the video lecture of this blog, go to this youtube video

Follow me on Twitter

Do comment your thoughts, questions or anything unique you learned below!

Users and Groups

When you run a process in Linux, you always run it as the owner of that process. Whatever files we have in our system, their access and ownership are permission-dependent.

Let's say, we have two users in the same Linux system: Adam and David. Adam can never access any of the files that belong to David and vice-versa. Every user has their own Home Directory where their user-specific files are stored.

If you open the terminal and enter the command pwd, it will give you a result like /home/lenovo

Here, lenovo is the username. If I had another user in this system named david, there would be a separate directory called /home/david. Only humans use usernames as a simpler way to identify the user.

But the Linux system uses a different way to identify a user. It stores a unique ID for every user called UID. The system also uses ‘groups’ to manage permissions. A group is a set of users with a similar set of permissions.

Imagine a scenario where you have a Linux system. Three types of employees use it in a company: 3 Web Developers, 2 DevOps Engineers, and 1 DBMS administrator. In such a situation, the admin can create groups for all categories of employees. Each group is assigned a unique GID, which helps the system identify the group.

Other Users

Even if you are the only human working with a Linux system, you are not the only user. There are other users as well, not necessarily a human. These users could be system daemons, which means processes that confirm everything runs smoothly and seamlessly in a system.

Root user

The most powerful user in the Linux system is the ‘root/superuser’. It can access any file and start/terminate any process in the entire system.

If you enter the command cat /etc/shadow in the terminal, it will show Permission denied. Because you, as a regular user, do not have sufficient permissions to view this file. If you enter ls -l /etc/shadow and check the detailed version of this file, you’ll find that this file belongs to the root user.

Sudo command

If you want to get the root access and read the /etc/shadow file, you need the help of a command known as sudo. It stands for ‘superuser do’. You can enter sudo cat /etc/shadow, and the system will ask for your password. And once you press enter after typing it, the terminal will display the file data.

If you want to use the terminal as a root user without typing sudo, you can use the command sudo su. As you can see below, all the commands you enter now will be as the root user. You do not need to write sudo or enter the password. Whatever commands you enter as the root user after sudo su, are not registered in the history.

Warning: Making yourself the root user and using the system is not recommended because you may make critical mistakes like deleting or destroying something important and crashing your system.

To exit the root environment, you can enter the command ‘exit’ and return to being a regular user.

Tip: You can check the list of all users that have permission to use the sudo command. Linux system stores the list in a file you can read using the command ‘sudo cat /etc/sudoers’. If you want a user to have or not have access to the sudo command, you can add or remove that user from this list.

Users other than you and sudo

As we understood earlier, there are many users in a Linux system. Two of them are you and root. Let’s talk about some others. If you enter the command sudo cat /etc/passwd, the terminal will show the list of all these users.

As you can see above, the users are like lenovo, daemon, mongodb, mysql etc. Each user in the list is responsible for running their specific processes. It might be different in your results for the same command.

/etc/passwd

Each line in the result of sudo cat /etc/passwd looks like this:

root:x:0:0:root:/root:/bin/bash

Each element of this line is separated by a colon. Let’s break this line down and understand all elements one by one.

1. Username - It will be different for every user. For a user named ‘lenovo’, it will be ‘lenovo’.
2. Password - This is ‘x’ here because the system stores it in another file ‘/etc/shadow’.
3. User ID (UID) - Every user in the system is assigned a unique user ID.
4. Group ID (GID) - Every group has a unique ID assigned to them by the system.
5. User info – Here it is ‘root’ based on the username. It can also be any contact detail or any other user info.
6. User’s Home Directory
7. Shell – Here it is root itself, but for a regular user like ‘lenovo’, it will be ‘/home/lenovo’Let’s talk about the ‘/etc/shadow’ file now.

/etc/shadow

If you enter the command sudo cat /etc/shadow, the terminal will display all users' password data.

Let’s take the first line containing the root user password details and understand each part. As you noticed, the last three values divided by a colon are empty. I'll explain them as well.

1. Username
2. Encrypted Password of the user – For the root user, this is ‘!’ but for a normal human user, it will be an encryption key.
3. Date of last password change – This is the number of days since 1st January 1970.
4. Minimum Password age – This is the number of days you have to wait to change the password. If it was 30, you would have to wait for 30 days to change the password again.
5. Maximum Password Age – This is the opposite of the above one. The number of days you can keep your password, after which you have to change it.
6. Warning Period – The number of days after the Max password age after which the system will start warning you to change your password.
7. Password Expiry Period – The number of days passed after password expiry. It is empty because 99999 days since 1970 have not passed yet.
8. Account Expiration Date – This is empty here as this is the root user. But for a regular user created in the system, its account expiration date can be customized at the time of creation.
9. Reserved Field – The dev community of Linux hasn’t yet figured out what to use this field for. If they figure it out in the future, they might fill or assign it with some value. But for now, it’s just an empty reserved field.

In most distros today, the authentication does not depend only on /etc/shadow. In the modern ecosystem, there is another system known as PAM (Pluggable Authentication Module).

/etc/group

Just like the above two files are for user and their password data. This one belongs to all the groups in the Linux system. If you enter the command sudo cat /etc/group, the terminal will show the group data the same way the above two files did.

Each group’s data is stored as lines with several line elements separated by a colon. Let’s break down the first line and understand.

1. Group name
2. Group password – There is generally no use for a group password. But if there is, you can create one to store it here.
3. GID (Group ID)
4. List of users – Here, it is empty, but in some groups, there can be one or more than one user, and their names are divided by commas.

Creating, Editing and Deleting a User

Create and Delete User

If you enter the command sudo useradd david, this will create a new user named ‘david’ in your Linux system along with a new /home/david directory and new entries in /etc/shadow, /etc/passwd, and /etc/group files. When you will restart your system, it will show you two users and ask which one you want to log in to.

To delete this user, you can enter the command sudo userdel david, and it will delete/undo everything you created, like a new directory and other user file data.

Changing Password

If you want to change a user named ‘david’ password, you can do it by entering sudo passwd david. This command will help you change the password, and even edit the other details in /etc/passwd we discussed like minimum and maximum password age etc.

Changing Hostname

In the user details shown in the terminal like lenovo@lenovo-Lenovo-ideapad-500-15ISK, lenovo-Lenovo-ideapad-500-15ISK is the hostname. You can change that with the command sudo hostname [new hostname].

Once you reopen the terminal after executing the command with the desired hostname, it will display the new one as it did above. In my case, I changed it to my Twitter handle 'gaurangcodes'.

Permissions

The whole above part of the blog was all about user management. Now let’s understand permissions. If you enter ls -l, the terminal will show a detailed version of all the files with various data elements. Let’s discuss each one, especially the first one which is most important.

1. Permissions
2. Username – The name of the user that owns the directory.
3. Group name – The group that owns the directory.
4. Last time the directory was modified.
5. Name of the directory

Let’s break down the first one, Permissions. Permissions always contain ten characters or bits. Divided into four types, are

1. Type – It tells you what type of file this is (everything in Linux is a file, even directories/folders). d represents ‘directory’. - represents a ‘file’ type with .extension. l represents ‘Link’.
2. User Permissions
3. Group Permissions
4. Other Permissions – If any other user gets access to these files, they will have this set of permissions.

These permissions are of 4 types: r, w, x, -

• r: readability
• w: writability
• x: executability
• '-' : Empty. No permission is granted

The format in which permissions are written is always rwx: readable writable executable. r, w, and x are the symbolic representations of permissions. If any one of them is replaced by - means the user does not have that particular permission. For example, r-x means a user can read or execute the file but cannot write it.

Just like rwx, ‘u, g & o’ are symbolic representations of the user, group and others. This will come in handy for changing the permission for any of them.

Changing Permissions

The chmod command stands for change or modify. You can change the permissions for any file or directory by executing it. For example, if I create a file testing.txt in a directory, it will not have 'write' permission for other users in the system, but I can change that by executing the command chmod o+w testing.txt.

As you can see, -rw-rw-r-- changed to -rw-rw-rw-.

+ symbol is used to add permission like, o+w

- symbol is used to remove permission like, o-w

You can also add/remove multiple permissions for multiple user types like chmod ugo-rwx testing.txt. But remember to maintain the order of ugo and rwx.

Numerical Representation of Permissions

Using ugo and rwx representations is not the only way to change permissions. As they are also represented by numbers where:

R is equivalent to 4

W is equivalent to 2

X is equivalent to 1

And

ugo (user, groups, others) can have 4+2+1 = 7 (rwx) permissions or 0+0+0 = 0 (---) permissions. Let’s say you want to give all permissions to all three u,g & o. For this, you can execute the command chmod 777 testing.txt where the order of numbers is maintained like, first for 'u', then for 'g', and last for 'o'. The permission subtotal can vary between 0 to 7 representing different combinations of permissions.

• 0 – No permission
• 1 – Execute
• 2 – Write
• 3 – Write + Execute
• 4 – Read
• 5 – Read + Execute
• 6 – Read + Write
• 7 – Read + Write + Execute

So between 0 to 7, all possible combinations of permissions covered that a user might need. But you need to maintain the order of u, g & o. In 777, the first digit is for the user, the second is for the group, and the third is for others.

chown and chgrp commands

Earlier, we saw how every file has an owner user. We can change that ownership for any file or directory by using the command sudo chown [username] [file/directory name]. chown command helps us change the ownership to any user we want.

There is also a group that owns that file or directory. You can change that by using sudo chgrp [group name] [file/directory name].

You can change both user and group ownership at the same time by using the command chown itself by using a colon to combine username and group name like, sudo chown newuser:someothergroup testing.txt.

umask command

When you create a new file or directory, it is assigned a default set of permissions for the user, group and others. You can change what default permissions they get by using the command ‘umask’ like umask 022. But this does not work like chmod. Instead, it subtracts 777 from 022 like 7-0 = 7, 7-2 = 5, 7-2 = 5. So, the set of permissions will be 755 and not 022.

suid, sgid, and sticky bit permissions

###suid suid stands for Set UID. As we all know, a regular user does not have access to the root files. And if you try to use sudo, the terminal will ask you for the password. If you are in an office environment, you might not know it and have to call the System Admin to enter it. If there is a root file you need to execute multiple times a day, it might not be possible to call the sysadmin to enter the password. In such situations, the ‘suid’ command comes in handy.

It will help you grant r, w, or x access to root files without using sudo in future. It means you can use them as the owner. ‘suid’ can be executed using either alphabetical (ugo and rwx) representation or numerical (777) representation.

Let's understand this with an example.

If we execute ls -l /etc/shadow, you can see that this file belongs to the root user. Now let’s try to execute this file with root privileges without using sudo and entering the password.

Just like rwx represent read, write and execute permissions. s represents special root privilege. If we execute sudo chmod u+s [filename] or sudo chmod 4755 [filename]. Here, s and the first digit of 4 represent that special root privilege that you will be granted.

As you can see, the permissions changed to -rwsr-xr-x. This s in the user section means you can execute the file now without sudo.

But what happens if you execute sudo chmod 4655 testing.txt? The 6 here means the execute permission digit (1) is not included for the user (4655 instead of 4755).

Now the small s is not there instead it’s capital S. This means you have the owner’s permission but still cannot execute the file because you entered 6 instead of 7, which omitted the execute permission.

sgid

Just like ‘suid’, there’s ‘sgid’ as well, represented by the digit 2. If you execute sudo chmod g+s [filename] or sudo chmod 2755 [filename], it will do the same thing but for groups.

As you can see, the s has shifted and permissions have changed to -rwxr-sr-x.

Sticky bit

The Sticky bit is the permission for deletion. It means any user may read, write, or execute a file, but only the owner can delete it and nobody else. Let’s understand it in detail.

If you execute ls -ld /tmp where we’re using -ld flag instead of -l, it will show the detailed info of the directory instead of the contents inside.

Did you notice something? The last x in permissions is not there, instead is replaced by t. This t is the sticky bit.

You can do this too by using the command sudo chmod +t [directory] or sudo chmod 1755 [directory] where 1 stands for, as you might have guessed, the sticky bit.

Earlier, we learned that 4 represents readability, 2 represents writability, and 1 represents executability. But when 4, 2, or 1 are prepended to the rwx numerical representation as we saw in the above examples,

4 becomes suid

2 becomes sgid

1 becomes sticky bit

As you can see, the last x has changed to t. This means only the owner user and the root (superuser) can delete this file.

Process Permissions

Suppose you have two human users in the system, 'larry' with a UID of 600 and ‘david’ with a UID of 500. The one other user is the ‘root’ with a UID of 0.

If user ‘larry’ tries to run the command sudo passwd david to change the password of user ‘david’, will he be able to change it? The answer is no, even though ‘larry’ is using the ‘sudo’ command. Let’s understand why.

When we execute a command, it is known as a process. There are three types of UIDs associated with any process.

• Effective UID – This belongs to the owner. If ‘larry’ is running the sudo passwd david command, he is running it as a root, so the effective UID will be 0.
• Real UID – This belongs to the person who has launched the process. Here it’s ‘larry’, so the Real UID will be 600.
• Saved UID – This UID switches between Effective and Real UID. Whenever you run a process, you never run it on higher privileges all the time, as you might end up messing up the system.

When you start your system, in most cases, the Effective UID and Real UID are the same because the s permission we understood earlier is not that common in the system. Most of the time, the Effective UID is the person that owns the file/directory.

Coming to the question, why can't ‘larry’ change the password of ‘david’? Because although the Effective UID is 0 (of the root) with which the process is working, the Real UID is 600 (of the person who launched the process), which is different from the UID of 500 that you’re trying to change the password of, so the system will not allow it.

But if you do it as the root user, the system will allow it because the root/superuser is very powerful and can bypass anything in the system.

Thank you for reading :)

To read the other blogs of this Linux Series, you can go to blog.gauranggaur.com

To see a video lecture of this blog, go to this youtube video

Follow me on Twitter here

Do comment your thoughts or anything unique you learned below!

Stream here means the transfer of data. That data is simple text.

Stdin (Standard Input)

Stdin has the code 0. Linux takes the standard input and gives an output. It can be a standard output or a standard error.

Stdout (Standard Output)

Stdout has the code 1. Standard output can be streamed at three places in your system:

1. Terminal Window
2. some file
3. Given to a pipe, which redirects it however you want.

If you want to store the text output in a file you can use the symbol >. For example: If you want to store the ls command output in an output.txt file. You can write the command ls > output.txt. If output.txt doesn’t exist, Linux will create it for you automatically.

Now, if you go to a different folder and want to store a command’s Stdout, you need to specify the path to output.txt like, ls > ~/output.txt

Notice that the previous entry of the ls output is overwritten with the new one. But you can avoid this and keep adding new entries without overwriting previous ones by using the symbol >> like ls >> output.txt

Notice that the previous Documents folder entry is still there while new entries are added. Controlled Stdout with less command

If you ever try to read a large file in the terminal, for example, with the command ‘cat /var/log/syslog’, it will populate your terminal with an extremely large output. It will become a hindrance for you to scroll up and down. To tackle this, you can do ‘less /var/log/syslog’, this will still display the file data in the terminal, but you can exit it by typing ‘q’ and your output will disappear as it is not populated in the terminal.

Stderr (Standard Error)

Stderr has the code 2. Let’s understand how Stderr is different from Stdout with an example. If you write the command lg > output.txt, Linux will give a standard error, but will it be saved in output.txt? Let’s see what happens.

As you can see, the output.txt file did get overwritten but is empty. Because a Stderr is not output. It is a Standard error. Output.txt expected a Stdout, hence it is empty.

But let’s say you still want to store Stderr to output.txt. For that, you can use 2> instead of > like ls 2> output.txt. We are writing 2 here because it denotes the Stderr code that Linux recognizes.

How to use the pipe to redirect Standard Output as Standard Input

Let’s say, you want to use the less command with ls -la /etc but if done so, it will give an error as both the output-giving commands are clashing with each other and less is confusing ls with a file/directory name.

You can use pipe ( | ) here to use both ls and less commands together like ls -la /etc | less and it will work just fine.

Let’s understand how did the above command work behind the scenes.

The ls -la /etc part of the command which is a Stdin gave a Stdout in response. But rather than populating on the terminal, it was transferred as a Stdin to the less part of the command by | (pipe)

Summary: Pipe allows you to take Stdout of one command and pass it as Stdin to another command.

Environment Variables

If you write echo $HOME, you’ll get something like /home/lenovo. If you do echo $USER, you’ll get lenovo.

Here HOME and USER are environment variables. In simple words, environment variables are useful information that the shell and its child processes use. You can check out all the environment variables stored in your system by using the command env

Environment variables are not constant. They keep changing based on what you’re doing in the system. For example, if you write the command echo $PWD where PWD is an environment variable, you’ll get an output like /home/lenovo. But if you change your present working directory with the cd command, the PWD data will change with it as well.

Now if you were wondering how the system gives out your location with the pwd command, it simply maintains an environment variable with that information and gives it out or uses it whenever the user desires.

Everything is a file in Linux, even the commands

If you type echo $PATH, you’ll find all the paths in which Linux checks for the command file that you have entered. If it doesn’t exist, you get the Standard error Command not found. The shell is saying that the file which executes that specific command is not found in any of the file paths stored in the PATH environment variable.

If you want to check the command files in one of these above paths, you can type any path like ls /usr/bin or for a better experience ls /usr/bin | less (to quit less environment, type ‘q’)

As you can see, the commonly used commands like ‘apt’, ‘ls’, and ‘npm’ exist as files in 'usr/bin'

Basic shell commands to interact with files

head command

If you type head [filename or filepath], it will show you the first ten lines that the file contains. If you want to see more or less lines, you can add the flag ‘-n’ and the number of lines like ‘head -n 5 /var/log/syslog’

tail command

You can do the same thing as head command but for the last ten lines. If you want to control the number of lines you want to display, use the flag -n with the desired number.

If you want to let the Stdout keep updating and populate your terminal, you can use the flag -f

As you can see, when I opened another terminal window, the terminal Stdout got updated with the information about the same (Read the third last line of Standard output).

sort command

This helps you sort the file content like below. You can also reverse sort the content with the flag -r

tr command

It takes a Standard input (Stdin) and translates or you can say, deletes and replaces it with a new, translated Standard Output (Stdout). You can use it with a pipe like cat output.txt | tr a-z A-Z

uniq command

Let’s say you have multiple lines with duplicate values, uniq command helps you display only the unique values as standard output. If you use -c flag, it will also show the number of occurrences of each value.

If you only want the non-repeated values, then you can use the flag -u. Only four values are not repeated in the example and hence, displayed. To see the opposite of this, which is only the duplicated values, you can use the flag -d

A problem you might face with uniq command

Let’s say the file contains values that are not adjacent to each other, then will the uniq command work?

The answer is no. As you can see, the repeated values are v, d, and r. But uniq command is not working on ‘v’ and ‘d’ unlike ‘r’. This is because r’s repetition happens in an adjacent way.

The solution

A handy solution is to use the sort command and transfer its Stdout as Stdin to uniq command via pipe like sort output.txt | uniq

Now you can see that ‘v’ and ‘d’ do not repeat in the Stdout.

wc command

wc command shows multiple count types of a file. If you write the command wc output.txt, it will give you three digits, which denote the number of lines, words and file size.

If you write wc -l [filename or path], you’ll get only the number of lines.

If you write wc -w [filename or path], you’ll get only the number of words.

If you write wc -c [filename or path], you’ll get only the size of the file in bytes.

grep command

It allows you to search files with the help of characters and regular expressions. For example, if you write env | grep PWD, you’ll get the search results with files that have PWD in their name. env command produces a standard output and pipe symbol (|) transfers it to the grep command as standard input.

Not only it shows you PWD, but also OLDPWD, which means the previous directory you were in. If you write cd - command, it will send you back to that directory (OLDPWD).

Thank you for reading :)

To read the previous blog of this Linux Series, click here

To see a video lecture of this blog, go to this youtube video

Follow me on Twitter here

Do comment your thoughts or anything unique you learned below!

The first thing your computer does after starting is to run the BIOS (Basic Input-Output System). In other words, it executes the software that is in BIOS. BIOS, also known as firmware, is installed by default in the ROM (Read-only Memory) chip in the CPU motherboard. That software checks if the system is working correctly or not. It will check if the memory is loading, if the RAM and Hard Disk are there, the devices connected to the system, whether the keyboard is connected, and many other preconditions to launching the Operating System.
Once the checks are over, it loads the Bootloader that starts the Operating System. A Bootloader is a computer program responsible for starting up the operating system.

What is an Operating System?

An Operating System or OS is the software responsible for managing the computing device and acts as an interface between the hardware and the user. The computing device can be a desktop, smartphone, ATM etc.
For software to be considered an OS, it needs a few pre-requisites:

1. Kernel: A Kernel is the core of any operating system with complete control over anything on the system. It facilitates the interaction between the software and the hardware.
2. File System: method or data structure that the OS uses to store and retrieve data in the memory.
3. User Interface (Command line interface or Graphical User interface)
4. Ability to manipulate data based on user commands

Why do Software Engineers prefer Linux or macOS over Windows?

Linux and macOS are very similar as they are both UNIX-based. UNIX is a multitasking, multiuser operating system designed for flexibility and adaptability. On the other hand, Windows derives most of its functionalities from other operating systems.

1. Linux is open source based. It makes feature updates and bug fixes a lot more frequent. If a developer finds a bug, s/he can raise the pull request. If it gets merged, it will be there in the next release. On the other hand, if a bug comes up in Windows, being a commercial product, the developers working on it make a list of all the bugs, fix them in one go and then come up with an update after a long time.
2. It supports almost all programming languages and offers a vast range of tools and applications useful for software development.
3. Linux Terminal is superior to Windows Command Prompt. The command line tools are developed natively for Linux, so it has better support.
4. Easier to get development-related things done in Linux. Like, as the ability to do Bash Scripting and SSH into some remote server.

Where did Linux come from? History of Linux

In 1969, two people, Ken Thompson and Dennis Ritchie developed the first Operating System called UNIX. A decade later, another person named Richard Stallman developed the GNU Project. Around that time, some other OS were also developed, like BSD and MINIX. All of them were similar to UNIX but lacked a unified Kernel. Then came Linus Torvalds in 1991, who started working on his side project while attending university. That project was Linux Kernel. Today this Linux Kernel is the unified kernel used by all Linux distributions that are out there, like Debian, Ubuntu, mintOS, fedora, arch Linux etc.

Terminal and Shell

You can access the terminal in your Linux distribution by pressing Ctrl+Alt+T. A Terminal is the text input/output environment that launches the shell. Now, what is a Shell? It is the command line interpreter. It executes every single command line-by-line, just like python does. You can say it is a program that takes your command from the keyboard and takes it to the kernel for execution. Most of the shells use Bash as their programming/scripting language.

As you can see in the screenshot, lenovo is the username of the computing device. You can choose whatever name you want on your computer. After @, lenovo-Lenovo-ideapad-500-15ISK is the hostname. The ~ symbol represents the home directory; it will change to the name of the directory you enter.

Some basic commands to interact with the OS

echo command

Whatever you write after echo, will be shown as the output. For example, echo Hello World will give the output ‘Hello World’

pwd command

It shows your present working directory or you can say, your location in the directory tree. The folders or directories are located in the system based on the tree structure. All directories have a parent directory. pwd command displays the path based on this tree's hierarchical structure. If you are in the Desktop folder, it will show /home/lenovo/Desktop as the output.

ls command

It shows all the directories and files that are present in the present working directory you’re currently in. The output of ls command depends on your location in the tree hierarchy. If you write ls -a as the command where -a is the flag and will show all the hidden files as well. If you add -l flag along with ls command, it will show many other details (like date of creation, permissions etc.) of every single item present in the directory. You can combine both flags by writing -la

cd command

It helps you change your location in the directory tree hierarchy.

By writing cd .. you can go back to a directory in the hierarchy. By using cd [directory name], you can go to that directory but it has to be present in your current directory. If it is not, then it will show not found output. Otherwise, you can also write a complete path to your desired directory after cd to tackle this problem, a path like pwd command shows. With cd - command you can go back to the previous directory. But this is not the same as cd .. command. cd .. takes you to the parent directory of the directory you were in. On the other hand, cd - takes you to the previous directory you were in, which is not necessarily the parent directory of the current directory as it is possible you switched to the current directory by putting cd [long path].

When you change the path, the exact hierarchy path gets added after the ~ sign which represents home.

mkdir and rmdir commands

mkdir command creates a new directory wherever you are situated in the hierarchy tree. Or wherever else you want by using the destination path and the -p flag like mkdir -p /new_folder/newDirectory, even if the new_folder does not exist, it will automatically create one for you.

rmdir command lets you remove the directory in the same way. But only if it does not have more directories inside it

If you want to remove a directory that has files and other subdirectories inside it, you have to use the command rm along with flag –r like rm -r new_folder or else the shell will give the error directory not empty. -r flag helps remove a directory recursively. To know exactly what rm –r folder command removed as the output, write rm –rv folder, where v means verbose.

touch command

This command creates a new file. You need to write the full name of the file with the extension like newfile.txt after touch like touch newfile.txt and this will create that desired file in whichever directory you are present.

cat command

It shows the content of the file, whatever is written in it.

history command

It lists all the commands you have entered in the shell since the beginning. If you want to use a command you used previously without typing everything once again, you can press the up key on your keyboard.

cp and mv commands

cp command will help you copy a file to any other folder with the help of its path. For example, if you write cp test.txt new_folder/subfolder1, this will copy test.txt to subfolder1.

mv command lets you do the same thing but rather than copying, it moves the file to the destination. This is like how you cut/paste in GUI.

If you want to copy or move a file without being present at the file’s location by using the cd command, you can write the whole path of the file instead like cp ~/Desktop/test.txt ~/Desktop/new_folder/subfolder1 (cp [file path] [destination path].

find command

It helps you find a file or directory in your system. Use it like ‘find [where to find] [argument] [file name]’, for example, find ~ -name file.txt. If you don’t know the extension of a file but know only the name then you can write file.* instead of file.txt. If you only remember the extension but not the name, you can write *.txt instead of file.txt. You can also use another flag -type if you want to specify the type of the element you’re searching for example –type d for directory (find ~ -type d -name new_folder2).

TIP: There are many flags for each command in Linux. You can’t possibly remember all of them. Whenever you need to find usable flags for a command, simply write in the shell ‘man [command name]’ where man means manual. Or you can also use google search like, 'mkdir command flags' and check out blogs or official documentation if you’re not comfortable with the ‘man’ command output.

To know what a command does or what is its purpose, you can write the command `whatis’ like whatis mkdir

To see a video lecture of this blog, go to this youtube video

Follow me on Twitter here

Do comment your thoughts or anything specific you learned below!

Let us understand the nature and role of the static keyword by looking at an example of a class and its instances. Like, the Phone class of a particular company.

public class Phone { 
    int ram; 
    string color; 
    int memory; 
    int price; 
    string model; 
    long unitSold; 

    public Phone (int ram, string color, int memory, int price, string model) { 
        this.ram = ram; 
        this.color = color; 
        this.memory = memory; 
        this.price = price; 
        this.model = model; 
        this.unitSold += 1; 
   } 
}

Please notice, the long unitSold; is a special property that will remain the same for all the phone units sold. The total number of phones sold by a company cannot be different for a particular phone model/version the same way as human population property remains the same for all humans. Every instance of this Phone class should increase the units sold by 1 digit with the help of code line this.unitSold += 1; inside the constructor.

Now, let us create some objects with this Phone class.

public class Main { 

    public static void main { 

        Phone galaxyS3 = new Phone (8, white, 45999, S3); 

        Phone galaxyS3Pro = new Phone (16, white, 65999, S3); 

        Phone galaxyS4 = new Phone (16, grey, 78999, S4); 



        System.out.println(galaxyS3.unitSold); 

        System.out.println(galaxyS3Pro.unitSold); 

        System.out.println(galaxyS4.unitSold); 
    }
}

Ideally, the result for all three print commands should be 3 in the console log because the company sold 3 smartphones in total. But that’s not the case.

        System.out.println(galaxyS3.unitSold);          //  1 

        System.out.println(galaxyS3Pro.unitSold);    //  1 

        System.out.println(galaxyS4.unitSold);          //  1

The properties that are not related to the objects and remain the same for all the class objects, we declare them as ‘static’. The static properties belong to the class rather than the instances of that class. Hence, we shouldn’t use the ‘this’ keyword for static variables in the constructor and replace it with the class name itself. So, we’ll use Phone.unitSold instead of this.unitSold

public class Phone { 

    int ram; 

    string color; 

    int memory; 

    int price; 

    string model; 

    static long unitSold; 



    public Phone (int ram, string color, int memory, int price, string model) { 

        this.ram = ram; 

        this.color = color; 

        this.memory = memory; 

        this.price = price; 

        this.model = model; 

        Human.unitSold += 1; 

     } 

}

Now the result for units sold will be 3 in the console log. Every time we create a new object, the unitSold value increases by 1.

        System.out.println(galaxyS3.unitSold);          //  3 

        System.out.println(galaxyS3Pro.unitSold);    //  3 

        System.out.println(galaxyS4.unitSold);          //  3

Static variables are not dependent on objects. Even without creating a single object, you can still access the static variable via Class.staticVariable or Phone.unitSold

Initialization of Static Variable

Initialization is the process of assigning a value to a variable. For example, int a; is declared, and we can initialize it by doing int a = 12;

Now for normal variables in a class, there are constructors to initialize the variables when new objects are created/instantiated like this.property = property;

But in the case of static variables, there are no objects. Here, the Static block comes into the picture. Static block only runs once and helps us initialize static variables inside a class.

Public class StaticBlock { 

    static int a = 4; 

    static int b;              // only declared, not initialized 


    static {                     // this is a static block 

        b = a*5;            //this will only run once when the program is loaded 

    } 


    public static void main (String[] args) { 

        StaticBlock obj = new StaticBlock (); 

        System.out.println (StaticBlock.a + “ and ” + StaticBlock.b);     // 4 and 20 

    } 

}

Static Methods

public static void main

This is the most familiar static method to anyone who has known java for a while. The question is, why this main method of the Main class is static?

This main method is the very first thing that is run by any java program. All the classes and objects have their references in the public static void main(){} function and are run through it. Without this main function, you cannot run any java program.

Let’s say you decide to remove the ‘static’ keyword and run the function as public void main(){} Java won’t be able to run this program because any function inside a class needs an object to run itself.

But it is impossible to create an object as any java program requires the main function to run first. Without it, you won’t even find the option to run the code. And if you can’t run the code, how will you create the object? And if you can’t create the object, how will you run the main function? This is the crazy paradox ‘static’ functionality helps us to break.

How? A static function doesn’t require an object to run itself. So, we can run the main function as the first function without an object by making it static.

Non-static method inside static

An example of non-static method inside a static one:

void greeting () { 

    System.out.println (“Hello world”); 

}

The above method can't be deployed inside a static method like

Public static void main { 

    void greeting () { 

    System.out.println (“Hello world”); 

    } 

} 

//OR

static void greets () { 

    greeting (); 

}

This will show an error and won’t run.

Why? Because inside any static method, you cannot run a non-static method. Anything non-static belongs to or requires an object to come into existence. And a static method does not depend on an object. Hence, you can’t have something inside static that does depend on an object.

How to deploy a non-static method inside a static method?

The problem we are facing: The non-static object greeting(); does not belong to an object.

The way to solve this problem: We can create a new object of the Main class and assign the greeting(); method to it.

static void greets () { 

    Main obj = new Main (); 

    obj.greeting(); 

}

Now the static method will accept the non-static greeting(); method as it belongs to an object now.

Static method inside non-static

An example of non-static method inside static:

static void greets () {       // this is a static function that 

    greeting (); 

} 



void greeting() {         // can be deployed inside non-static greeting() function 

    greets(); 

}

This is possible without any error because a static method does not depend on any object and is independent. Hence, it can be deployed anywhere in the program.

Conclusion

Any static variable or method belongs to the class and not to the instances of that class. Anything static is independent of objects and does not need to assign itself to an object. Hence, static variables or functions can be placed anywhere in a java program, including all non-static objects or functions.

What are Classes and Objects?

Class is a template that Objects use to come into existence. An Object’s properties and functions are all derived from its Class.

Let’s understand with the help of an example:

If you want to store data of 5 employees. For salary numbers, you can do it in an array, like int [] salary = new int [5]; Now you want to store employee names, you can do String [] name = new String [5]; And for years of experience, you can do float [] exp = new float [5]; But how do we combine all three above data types into a single entity, a custom datatype? We can do it via Class.

class Employee { 
    String name;  
    int salary; 
    float exp; 
}

How do we create multiple Objects of different employees by using this Class?

Before that, we need to understand how do we access the Class properties as all Employee Objects need these properties to come into existence and store data about themselves. We access/bind them with the Dot operator.

The Role of Dot Operator

Dot Operator links the reference variable (employee1, employee2, employee3) with the instance variable (name, salary, exp).

For example, employee1.name will represent Michael.

But there’s a problem. If you declare Employee employee1; and type System.out.println(employee1.name); The console should have shown ‘null’ as the default value. Instead, the compiler will give an error that variable employee1 has not been initialized and does not acquire any space in the heap memory.

Here the ‘new’ keyword comes in.

The Role of ‘new’ Keyword

To create the Objects, you need the ‘new’ keyword. The ‘new’ keyword initializes, means allocates memory for the Object at runtime.

Employee employee1 = new Employee ();

Now if you type System.out.println(employee1.name); it will print ‘null’ in the console, which is the default value of any String variable. Now we can successfully allocate value with the dot operator.

employee1.name = “Michael”; 
employee1.salary = 150000; 
employee1.exp = 2.5f; 

employee2.name = “Emily”; 
employee2.salary = 180000; 
employee2.exp = 3.5f;

But did you notice this process of binding is quite repetitive? We have to write employee1 again and again. We can use a shortcut and allocate data to their respective instance variables (like Michael to name and 150000 to salary).
Here the Constructor comes in.

The Role of a Constructor

A Constructor is a special built-in type of Function in the Class that runs and defines what happens when we create an Object. We can allocate the data by writing it in the argument of a constructor function. If we don’t write the data, the constructor will bind default values (like null, 0 and 0.0) to the Object.
For Constructor to bind the values written in the brackets like new Employee (“Michael”, 150000, 2.5); we need to write the Constructor function inside the Class like:

class Employee { 
         String name;  
         int salary; 
         float exp; 

         Employee () { 
                 employee1.name = “Michael”; 
                 employee1.salary = 150000; 
                 employee1.exp = 2.5f; 
         } 
}

But this will give you an error. Class is a template to create multiple Objects. You can’t mention only one employee or custom object inside the Class. You won’t make a separate constructor for each employee as that would be counterproductive and repetitive.

We need a way to add values to properties of multiple Objects with the same Constructor. We need one word to access every Object and bind the Constructor argument with the Class properties. So that the Constructor can automatically replace that word like word.name with Object names like employee1.name, employee2.name or employee3.name.

Here the keyword ‘this’ comes into the picture.

The Role of ‘this’ keyword

The ‘this’ keyword helps us bind the arguments of a Constructor function like new Employee (“Michael”, 150000, 2.5); with the respective properties like name, salary, exp present inside the Class to be inherited by all Objects.
With the ‘this’ keyword, we can create as many employee Objects as we want without repeatedly writing code like employee1.name = “Michael”, employee2.name = “Emily”, employee3.name = “Ryan”.

class Employee { 
             String name;  
             int salary; 
             float exp; 

             Employee (String name, int salary, float exp) { 
                             this.name = name; 
                             this.salary = salary; 
                             this.exp = exp; 
             } 
}

In the code above, this.name = name; doesn’t have to be that way. You can write a different argument name in the Constructor, for example:

class Employee { 
          String name;  
          int salary; 
          float exp; 

          Employee (String empName, int salaryOfEmployee, float expYears) { 
                          this.name = empName; 
                          this.salary = salaryOfEmployee; 
                          this.exp = expYears; 
          } 
}

We can also pass desired custom functions inside the Class like:

class Employee { 
        String name;  
        int salary; 
        float exp; 

        void greeting () { 
              System.out.println (“Hello everyone, my name is ” + this.name);
        }

        Employee (String empName, int salaryOfEmployee, float expYears) { 
                this.name = empName; 
                this.salary = salaryOfEmployee; 
                this.exp = expYears; 
        } 
}

If you write employee1.greeting(); it will print “Hello everyone, my name is Michael”. If you write employee2.greeting(); it will print “Hello everyone, my name is Emily”.

Conclusion

From the series of events above, we understood that the 'this' keyword is the last step in the series that helps us create unlimited Objects with the help of a Class and its Constructor function. The 'this' keyword replaces itself with the Object name like employee1, employee2, employee3 and binds it to the Class Properties like name, salary, experience with the help of dot operator. We can use the 'this' keyword in self-made, custom functions inside the Class for attaching the different assigned values to Class properties for various Objects as we saw in the greeting(); function.

Thanks for reading! :)

A programming language is a language used by humans to interact with computers. Programmers assign tasks to machines by writing lines of code on what, why, when and how to do something. It can be anything. The possibilities are endless.

There are two main types of programming languages. Humans use High-level programming language to write tasks and machines use the low-level programming language to understand the assigned task.

Low-Level Programming Languages

Low-Level Programming Languages are almost impossible to be understood by most programmers. They come into existence after the compilation of the programmer-written code to be understood by computer machines. Now there are two types of low-level programming languages:

Assembly Language

Assembly language is a type of low-level language that is writable by humans but much closer to the understanding of computers. It is much faster to execute by the processor but is comparatively more difficult to write than high-level languages like c++, java etc.

Unlike English words like print and class, in Assembly language, we use mnemonics like LDA and STA that contain instructions like load and store. These mnemonic instructions get translated to machine language by the Assembler. Programmers wrote Assembly language frequently during the early days of programming. But now, it is done only when the performance and speed are of the highest importance for a software task or if the hardware is outdated.

Machine Language (Binary Code)

Everyone is most familiar with this computer language. Thanks to the dropping green 0s and 1s in Matrix. Machine or Binary Language is hexadecimal sets of 16 0s and 1s that represent every single instruction and data fed to the computer by humans. Unlike Assembly and High-level languages, it does not need an Assembler or Compiler to be understood by the machine and is the closest to a computer’s understanding.

0101000000010000 : Add sum to the acc;
1011000000001010 : Jump to done if n < 0;

High-Level Programming Languages

It will be almost impossible to write instructions for the computer in 0s and 1s or mnemonics, so we created High-Level Programming Languages. They are human-friendly programming languages containing English words as commands. Programmers can turn their ideas into executable, instruction-based code and design everyday software and applications. They are executed by being translated to machine language by a compiler. There are two ways we can distinguish high-level programming languages.

Procedural Programming Language

This category comes from the initial days of programming. For a procedural programming language, the procedure is the priority and data come second. It uses a top-down approach where we break down a big problem into successively smaller chunks. It is easy to track the flow of a program and find bugs due to the well-structured, systematic order of statements.

Examples of Procedural Programming Languages: Pascal, COBOL, C, Basic, Fortran, Python

Functional Programming Language

Functions are a block of code with a set of instructions/calculation/solutions that we can repeatedly use by using the function’s name instead of writing all of the code once again. Functional Programming languages are the ones that treat functions as first-class citizens in their environment. It means the Functional language treats them like any other variable. We can pass functions as arguments in other functions, return them in a higher-order function, and assign them as a value to another variable.

Examples of Functional Programming Languages: JavaScript, Python, Clojure, Elixir, Scala, Erlang, Haskell, Common lisp, Racket, SQL

Object-Oriented Programming Language

Object-oriented programming language revolves around objects. When we turn a dataset into code, it is known as an object. Let me explain with an example. An object ‘student’ consists of a name (string), roll number (integer), and percentage (float). Now, we can use the ‘student’ object as a custom datatype everywhere in the code.

Classes are templates, and Objects are their instances. In simple words, we create Objects using classes. Class is a template that Objects (like students) use for their structure. Classes can also contain functions that manipulate the data to produce desired results.

For example, if a Class (template) has a Function that decreases roll number by one digit, then all the Objects inherited (created) from that class will have this function and reduce their roll numbers by 1.

Object-oriented programming follows the bottom-up approach. Contrary to the top-down approach, we define small components and solve problems at a fundamental level. We link them to build elements and repeat the integration until we have a complete system/solution. This programming approach is the easiest and most efficient to solve real-world problems as every problem is treated as an entity.

Examples of Object-oriented Programming Languages: Java, C++, C#, JavaScript, Python, PHP, Ruby, Swift, Kotlin, Scala, Common Lips

The doubt you might have

In the examples, you may have noticed that some languages like Python and JavaScript are in multiple categories. So, the question is, do these programming languages support more than one approach to programming? The answer is yes! Most modern High-level programming languages are hybrid in structure and designed for general-purpose programming.

Static and Dynamic Programming Languages

Here you won’t find the programming languages in both categories. If a programming language is static by nature, it remains Static and vice versa. In the previous section of High-Level programming languages, we talked about the structure of the code and programming languages can support multiple such approaches. But here, it is based on compile-time and runtime.

Static Programming Languages

In statically typed programming languages, we specify the data type of the variable. For example, int a = 12; or string wealth = “cars”;. We do this because the compiler checks all details of variables. And a variable cannot have two data types, or the code won’t compile and convert to machine language in a Static Programming Language System.

For example, if the variable string wealth = “cars”;, then we can’t have int wealth = 24; at the same time because the compiler will show an error.

The error will also show an error if we write int wealth = “car”; because here the datatype declared is an integer but, the value assigned is a string. The compiler will show the syntax error before compiling.

The Advantages of using Static programming languages

We have more control over the code. We can remove all syntax errors even before the source code goes for compilation. That saves a lot of time and energy that may go into debugging. Static programming languages are comparatively faster at runtime as the compiler translates all the code before running.

Examples of Static Programming Languages: Java, C++, C#, C, Pascal, Go, Kotlin, Swift, Haskell, Fortran, Scala

Dynamic Programming Languages

In Dynamically typed programming languages, we do not specify the variable datatype. For example, we can write like a = 12 or wealth = “cars”. In a Dynamic Programming Language, the datatype checking happens at runtime, when the program is running as machine code in the form of 0s and 1s.

If you write a = 3 and a = “word” in the same code, a Dynamic Programming Language will not show an error when compiling. The computer will show an error when the code is running (at runtime as machine code).

The Advantages of using Dynamic Programming Languages

We have to write a lot less code comparatively as we don’t have to deal with datatype specifications. But might give errors at runtime. But that does not bother programmers as dynamic programming languages are faster to write, easy to understand and more flexible. That makes bug finding easier.

Examples of Dynamic Programming Languages: Python, JavaScript, Ruby, PHP, Ruby, Lua, Perl

Thanks for reading! Let me know what you understood and what you didn't. Let's connect in the comment section :)