Taogen's Blog

Write Statistical SQLs

Before write the code, you can write statistical SQLs first. Because the core of statistical APIs are SQLs. As the saying goes, “first, solve the problem, then, write the code”.

Define Parameter and Response VOs

VO (value object) is typically used for data transfer between business layers and only contains data.

Parameter VOs

@Data
public class SomeStatParam {
  private Date beginTime;
  private Date endTime;
  private Integer type;
  private Integer userId;
  ...
}

Response VOs

@Data
public class someStatResult {
  ...
}

Write the APIs

@RestController
@RequestMapping(value = "/moduleName")
public class SomeStatController {
    @GetMapping(value = "/getSomeStat")
    public ResponseEntity<SomeStatResult> getSomeStat(SomeStatParam someStatParam) {
        SomeStatResult data = someStatService.getSomeStat(someStatParam);
        return ResponseEntity.ok(data);
    }
}

Test APIs in Your Local Environment

Before integrating third-party APIs in your web application, it’s better to test the APIs with an API tool like Postman to ensure the APIs work properly in your local environment. It can also let you get familiar with the APIs.

Add API Configurations to Your Project

There are some common configurations of third-party APIs you need to add to your project. For example, authorization information (appId, appSecret), API URL prefix, and so on.

Add Configurations

Add the configurations to your spring boot configuration file application.yaml:

thirdParty:
	thirdParty1:
		appId:
		appSecret:
		apiBaseUrl:
	thirdParty2:
	  	...

Define the Configuration Bean

Add a spring bean to receive the configurations:

YourThirdPartyConfig.java

@Configuration
@EnableConfigurationProperties
@ConfigurationProperties(prefix = "thirdParty.yourThirdPartyName")
@Data
public class YourThirdPartyConfig {
    protected String appId;
    protected String appSecret;
    protected String apiBaseUrl;
}

You can access the configurations by @Autowired it

@Autowired
private YourThirdPartyConfig yourThirdPartyConfig;

String appId = yourThirdPartyConfig.appId;
String appSecret = yourThirdPartyConfig.appSecret;
String apiBaseUrl = yourThirdPartyConfig.apiBaseUrl;

Define Response Code

public class YourThirdPartyErrorCode {  
    public static final Integer SUCCESS = 0;
    public static final Integer INVALID_APPID = 1;
    public static final Integer INVALID_TOKEN = 2;
}

Define API URLs

public class YourThirdPartyApiUrl {
    public static class Module1 {
        public static final String xxx_URL = "";
    }
    
    public static class Module2 {
        public static final String xxx_URL = "";
    }
}

Define Base Service

@RequiredArgsConstructor
public abstract class YourThirdPartyBaseService {
    protected final YourThirdPartyConfig yourThirdPartyConfig;

    private final RestTemplate restTemplate;

    public ObjectNode getRequest(String url, MultiValueMap params) throws IOException {
        UriComponentsBuilder uriBuilder = UriComponentsBuilder.fromHttpUrl(
                yourThirdPartyConfig.apiBaseUrl + url)
            .queryParams(params);
        ResponseEntity<String> response = restTemplate.getForEntity(uriBuilder.toUriString(), String.class);
        if (response.getStatusCode().is2xxSuccessful()) {
            ObjectNode jsonObject = JacksonJsonUtil.jsonStrToJsonObject(response.getBody());
            if (YourThirdPartyErrorCode.SUCCESS.equals(jsonObject.get("code").asInt())) {
                return jsonObject;
            } else {
                throw new RuntimeException("Failed to get access token: " + jsonObject.get("message").asText());
            }
        } else {
            throw new RuntimeException("Failed to get access token: HTTP error code " + response.getStatusCodeValue());
        }
    }
}

Get and Cache Access Token

@Service
@Slf4j
@RequiredArgsConstructor
public class YourThirdPartyTokenService extends YourThirdPartyBaseService {
    public static final String KEY_ACCESS_TOKEN = "yourThirdParty:%s:access_token";
    private final YourThirdPartyConfig yourThirdPartyConfig;
    private final RestTemplate restTemplate;
    private final RedisUtil redisUtil;
    
    public String getAccessToken() throws IOException {  
        return getAccessToken(yourThirdPartyConfig.appId, yourThirdPartyConfig.appSecret);  
    }

    public String getAccessToken(String appId, String appSecret) throws IOException {
        String accessToken = getAccessTokenFromCache(appId);
        if (StringUtils.isNotBlank(accessToken)) {
            log.debug("get access_token from redis: {}", accessToken);
            return accessToken;
        }
        log.debug("get access_token from API");
        ObjectNode jsonObject = getTokenInfoFromApi(appId, appSecret);
        setTokenToCache(jsonObject, appId);
        return jsonObject.get("accessToken").asText();
    }

    private String getAccessTokenFromCache(String appId) {
        return redisUtil.get(String.format(KEY_ACCESS_TOKEN, appId), 0);
    }

    private void setTokenToCache(ObjectNode jsonObject, String appId) {
        String accessToken = jsonObject.get("result").get("accessToken").asText();
        Integer expiresIn = jsonObject.get("result").get("expires").asInt();
        redisUtil.set(String.format(KEY_ACCESS_TOKEN, appId), accessToken, 0, "nx", "ex", expiresIn - 60);
    }


    public ObjectNode getTokenInfoFromApi(String appId, String appSecret) throws IOException {
        MultiValueMap<String, String> params = new LinkedMultiValueMap<>();
        params.put("appId", Arrays.asList(apiKey));
        params.put("appSecret", Arrays.asList(secretKey));
        return super.getRequest(YourThirdPartyApiUrl.Token.GET_ACCESS_TOKEN_URL, params);
    }
}

Write Code

Define the API Interfaces

Here we use the example of the social login API service to demonstrate.

public interface SocialLoginService {
  JSONObject func1(JSONObject params);
  JSONObject func2(JSONObject params);
  // More functions...
}

Why use JSONObject as parameters and return types of methods of the interface?

Different third-party API service platforms have different data structures. The parameters and return types of methods need to be extensible.

Define Constants

Keys for API parameters

public class Auth0Key {
  public static final String CODE = "code";
  public static final String MSG = "msg";
  public static final String USER_NAME = "user_name";
}

API request URIs

public class Auth0Url {
  public static final String LOGIN = "/login";
  public static final String GET_USER_INFO = "/user/getInfo";
  // More URLs ...
}

Write the API Implementing Class

@Service
public class Auth0SocialLoginService implements SocialLoginService {
  @Autowired
  private Auth0Config auth0Config;
  
  public JSONObject func1(JSONObject params) {
    ...
  }
  public JSONObject func2(JSONObject params) {
    ...
  }
}

Common third-party APIs and their class names

Write Parameter Builders and Result Handlers

Parameter builders

public interface SocialLoginParamBuilder {
	JSONObject getFunc1Param(Xxx entity);
	JSONObject getFunc2Param(Xxx entity);
}

@Component
public class Auth0SocialLoginParamBuilder implements SocialLoginParamBuilder {
	JSONObject getFunc1Param(Xxx entity) {...}
	JSONObject getFunc2Param(Xxx entity) {...}
}

Result handlers

public interface SocialLoginResultHandler {
	Vo handleFunc1Result(JSONObject result);
	Vo handleFunc2Result(JSONObject result);
}

@Component
public class Auth0SocialLoginResultHandler implements SocialLoginResultHandler {
	Vo handleFunc1Result(JSONObject result) {...}
	Vo handleFunc2Result(JSONObject result) {...}
}

Use Your Interface

@Autowired
@Qualifier("auth0SocialLoginService")
private SocialLoginService socialLoginService;
// or
@Autowired
private SocialLoginService auth0SocialLoginService; // The SocialLoginService object name must same as the implementing class name and the first character should be lowercase.

@Autowired
@Qualifier("auth0SocialLoginParamBuilder")
private SocialLoginParamBuilder socialLoginParamBuilder;

@Autowired
@Qualifier("auth0SocialLoginResultHandler")
private SocialLoginResultHandler socialLoginResultHandler;

JSONObject params = auth0SocialLoginParamBuilder.getFunc1Param(entity);
JSONObject result = auth0SocialLoginService.func1(params);
Vo vo = auth0SocialLoginResultHandler.handleFunc1Result(result);

This post will cover how to create a static website using VitePress.

Before using VitePress you must install Node.js v18 or higher.

Initialize VitePress Project

Add vitepress dependency

$ mkdir my-site
$ cd my-site

# add vitepress to devDependencies
$ npm add -D vitepress
# or
$ yarn add -D vitepress

VitePress is used in the development process as a build tool. It converts your Markdown files to HTML files. You don’t need VitePress in runtime.

Scaffold a basic project

$ npx vitepress init

npx can run your installed package directly, you don’t need to add any npm script to your package.json. You also can do npx package_command by using npm run your_script.

You need to set some basic configuration for your website.

• Setting the VitePress directory. ./ means using the root directory.
• Setting your site title.
• Setting your site description.
• Other configurations just use the default settings.

┌  Welcome to VitePress!
│
◇  Where should VitePress initialize the config?
│  ./
│
◇  Site title:
│  My Awesome Project
│
◇  Site description:
│  A VitePress Site
│
◇  Theme:
│  Default Theme
│
◇  Use TypeScript for config and theme files?
│  Yes
│
◇  Add VitePress npm scripts to package.json?
│  Yes
│
└  Done! Now run npm run docs:dev and start writing.

Running the project

Start a local server

npm run docs:dev
# or
yarn docs:dev
# or
npx vitepress dev

Visiting http://localhost:5173 to access the website

Git Configurations

Initialize the git repository

$ cd my-site
$ git init .

Config gitignore

Add the .vitepress/cache directory to .gitignore

.vitepress/cache

Result .gitignore

.idea

# macOS
.DS_Store

# vitepress
.vitepress/cache

# Dependency directories
node_modules/

# build output
dist

VitePress Configurations

Site Config

The site configuration file is .vitepress/config.mts

export default defineConfig({
	title: '{my_title}',
	description: '{my_description}',
	srcDir: 'src',
	srcExclude: [
		'someDir/**',
		'someFile',
	],
	// Whether to get the last updated timestamp for each page using Git.
	lastUpdated: true,
	head: [
		['link', {rel: 'shortcut icon', type: "image/jpeg", href: '/logo.jpeg'}],
		// These two are what you want to use by default
		['link', {rel: 'apple-touch-icon', type: "image/jpeg", href: '/logo.jpeg'}],
		['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "72x72", href: '/logo.jpeg'}],
		['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "114x114", href: '/logo.jpeg'}],
		['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "144x144", href: '/logo.jpeg'}],
		['link', {rel: 'apple-touch-icon-precomposed', type: "image/jpeg", href: '/logo.jpeg'}],
		// This one works for anything below iOS 4.2
		['link', {rel: 'apple-touch-icon-precomposed apple-touch-icon', type: "image/jpeg", href: '/logo.jpeg'}],
	],
	themeConfig: {
		//...
	}
})

$ mkdir src/ && mv index.md src/

Files

• src/public/logo.jepg

srcDir

Move the home page index.md to src/index.md

$ mkdir src/
$ mv index.md src/

export default defineConfig({
  
  srcDir: 'src',
  
})

srcExclude

Optional config. If you need to add excluded directories and files.

export default defineConfig({
  
  srcExclude: [
    'someDir/**',
    'someFile',
  ],
  
})

For example:

• docs/**,
• index-en.md

lastUpdated

export default defineConfig({
  
  // Whether to get the last updated timestamp for each page using Git.
  lastUpdated: true,
  
})

Head

Shortcut icon

export default defineConfig({

	head: [
		['link', {rel: 'shortcut icon', type: "image/jpeg", href: '/logo.jpeg'}],
		// These two are what you want to use by default
		['link', {rel: 'apple-touch-icon', type: "image/jpeg", href: '/logo.jpeg'}],
		['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "72x72", href: '/logo.jpeg'}],
		['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "114x114", href: '/logo.jpeg'}],
		['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "144x144", href: '/logo.jpeg'}],
		['link', {rel: 'apple-touch-icon-precomposed', type: "image/jpeg", href: '/logo.jpeg'}],
		// This one works for anything below iOS 4.2
		['link', {rel: 'apple-touch-icon-precomposed apple-touch-icon', type: "image/jpeg", href: '/logo.jpeg'}],
		],

})

src/public/logo.jepg

Theme Config

The theme configuration file is also .vietpress/config.mts

import {type DefaultTheme, defineConfig} from 'vitepress'

export default defineConfig({
  themeConfig: {
    nav: nav(),
    sidebar: {
	    '/example/': sidebarExample()
    },
    
    logo: {src: '/your-logo.png', width: 24, height: 24},
    search: {
      provider: 'local'
    },
    outline: {
      level: "deep"
    },
  }
})

function nav(): DefaultTheme.NavItem[] {
    return [
        {text: 'Home', link: '/'},
        {text: 'Example', link: '/example/'},
    ];
}

function sidebarExample(): DefaultTheme.SidebarItem[] {
    return [
		{text: 'Examples', link: '/examples.md'},
	]
}

Files

• src/public/your-logo.png

Logo

export default defineConfig({
  themeConfig: {
    
    logo: {src: '/your-logo.png', width: 24, height: 24},
    
  }
})

src/public/your-logo.png

Local Search

export default defineConfig({
  themeConfig: {
    
    search: {
      provider: 'local'
    },
    
  }
})

Outline

export default defineConfig({
  themeConfig: {
    
    outline: {
      level: "deep"
    },
    
  }
})

Home Page Config

Config home page in src/index.md

Image

hero:
  image:
    src: your-home-image.png
    alt: your-home-image

src/public/your-home-image.png

Feature Icons

features:
  - icon: 🚀
    title: 
    details: 

Other Config

Favicon

Put your website icon to src/public/favicon.ico

Deployment

Create a repository on GitHub

Create button on the right top bar of GitHub -> New repository

Enter your repository name. For example, my-site

Click the “Create Repository” button.

Commit your local repository to GitHub

git add .
git commit -m '🎉feat: First commit'
git remote add origin git@github.com:{your_username}/{repo_name}.git
git branch -M main
git push -u origin main

Deploy your project on Vercel

Go to Vercel website.

Click the “Add New” button -> Project -> Import Git Repository

Add permission to access the new GitHub repository to Vercel

After finishing the permission configuration, you can find the new GitHub repository on Vercel.

Select the repository. And click “Import”

Override “Build and Output Settings”

• Build Command: npm run docs:build or yarn docs:build
• Output Directory: .vitepress/dist

Click “Deploy”

After the deployment process is finished, you can visit your website by Vercel provided URL.

Common text processing commands on Linux

• grep
• sed
• awk
• tr
• sort
• wc

Filter

Filter lines

echo -e "Foo\nBar" | grep "Fo"

Insert

Insert lines

Insert to the specific line

echo -e 'Foo\nBar' | sed '2i\the new line\' # Foo\nthe new line\nBar

Add line to beginning and end

echo -e 'Foo\nBar' | sed '1 i\First line'
echo -e 'Foo\nBar' | sed '$aEnd line'

Insert lines after match pattern

echo -e 'Foo\nBar' | sed '/Foo/a NewLine1\nNewLine2'
echo -e 'Foo\nBar' | sed '/Foo/r add.txt'

Insert text to the beginning and end

Insert text to the beginning of each line

echo 'foo' | sed 's/^/BeginText/'

Insert text to the end of each line

echo 'foo' | sed 's/$/EndText/'

Insert text to the begining and end of each line

echo 'foo' | sed 's/^/BeginText/' | sed 's/$/EndText/'

Insert a new line to the end of each line

echo -e 'Foo\nBar'| sed 's/$/\r\n/'

Replace

Replace first

echo "old names, old books" | sed 's/old/new/'
# or
echo "old names, old books" | sed '0,/old/{s/old/new/}'

Replace all

echo "old names, old books" | sed 's/old/new/g'

Remove

Remove matched lines

echo -e "Foo\nBar" | sed '/Foo/d'

Remove empty line

echo -e "Foo\n \nBar" | sed '/^\s*$/d'
# or
echo -e "Foo\n \nBar" | sed '/^[[:space:]]*$/d'

Remove comment /**/ or //

# reomve lines start with / or *
sed '/^ *[*/]/d' 

Remove n lines after a pattern

# including the line with the pattern
echo -e "Line1\nLine2\nLine3\nLine4" | sed '/Line1/,+2d' # Line4

# excluding the line with the pattern
echo -e "Line1\nLine2\nLine3\nLine4" | sed '/Line1/{n;N;d}' # Line1\nLine4

Remove all lines between two patterns

# including the line with the pattern
sed '/pattern1/,/pattern2/d;'
echo -e "Foo\nAAA\nBBB\nBar\nCCC" | sed '/Foo/,/Bar/d' # CCC

# excluding the line with the pattern
sed '/pattern1/,/pattern2/{//!d;};'
echo -e "Foo\nAAA\nBBB\nBar\nCCC" | sed '/Foo/,/Bar/{//!d;}' # Foo\nBar\nCCC

Substring

Get substring by index

cut -c start-end1,start_end2
printf 'hello,world' | cut -c 1-5 # hello

Split and get fields

cut -d DELIMITER -f field_number1,field_number2
print 'hello,world' | cut -d ',' -f 2 # world

Find String

Find String by Pattern

echo -e 'Hello Java developer!\nHello Web developer!' | sed 's/Hello \(.*\) developer!/\1/'

Join

Join lines

echo -e "Foo\nBar" | tr '\n' ' '

Split

Split to multiple lines

echo "Foo Bar" | tr '[:space:]' '[\n*]'

Deduplication

# sort and deduplication
echo -e "1\n3\n2\n1" | sort -u

Sort

echo -e "1\n3\n2\n1" | sort

Count

Count lines

echo -e "1\n3\n2\n1" | wc -l # 4

Count matched lines

echo -e "1\n3\n2\n1" | grep -c "1" # 2

Count matched number of string

echo "hello world" | grep -o -i "o" | wc -l # 2

Format

To Upper/Lower Case

# to upper case
echo "hello WORLD" | tr a-z A-Z
# to lower case
echo "hello WORLD" | tr A-Z a-z

Format JSON string

echo '{"name":"Jack","age":18}' | jq .
echo '{"name":"Jack","age":18}' | jq .name

Crypto

Encode

Base64

Base64 Encode

printf 'hello' | base64
# or
echo -n 'hello' | base64

• -n: do not output the trailing newline

Base64 Decode

printf 'hello' | base64 | base64 -d

URL encode

URL encode

printf '你好' | jq -sRr @uri
# or
echo -n '你好' | jq -sRr @uri

• -n: do not output the trailing newline

Hash

md5

md5 -r /path/to/file

printf 'hello' | md5sum

echo -n 'hello' | md5sum
# or
echo -n 'hello' | md5

• -n: do not output the trailing newline

md5sum on linux, md5 on macOS

sha

shasum -a 256 /path/to/file

printf 'hello' | shasum -a 256

Examples

Wrap in double quotes and join with comma

echo 'hello
world' | sed 's/^/"/' | sed 's/$/"/' | tr '\n' ','

SSH, or secure shell, is an encrypted protocol used to manage and communicate with servers. You can connect to your server via SSH. There are a few different ways to login an SSH server. Public key authentication is one of the SSH authentication methods. It allows you to access a server via SSH without a password.

Creating SSH keys

List supported algorithms of SSH keys on your client and server:

$ ssh -Q key

Output

ssh-ed25519
ssh-ed25519-cert-v01@openssh.com
ssh-rsa
ssh-dss
ecdsa-sha2-nistp256
ecdsa-sha2-nistp384
ecdsa-sha2-nistp521
ssh-rsa-cert-v01@openssh.com
ssh-dss-cert-v01@openssh.com
ecdsa-sha2-nistp256-cert-v01@openssh.com
ecdsa-sha2-nistp384-cert-v01@openssh.com
ecdsa-sha2-nistp521-cert-v01@openssh.com

Choose an algorithm that supports both your client and server for generating an SSH key pair.

$ ssh-keygen -t {your_algorithm} -C "{your_comment}"

We recommend using the ed25519 algorithm to generate your SSH key. The Ed25519 was introduced on OpenSSH version 6.5. It’s using elliptic curve cryptography that offers better security with faster performance compared to DSA, ECDSA, or RSA. The RSA is even considered not safe if it’s generated with a key smaller than 2048-bit length.

$ ssh-keygen -t ed25519 -C "mac_taogenjia@gmail.com"

Output

Enter file in which to save the key (/Users/taogen/.ssh/id_ed25519):
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in {filename}
Your public key has been saved in {filename}.pub

You can specify your SSH key’s filename. If you don’t want to change the filename, by default the private key filename is id_{algorithm} and the public key filename is id_{algorithm}.pub .

For security reasons, it’s best to set a passphrase for your SSH keys.

Coping the SSH public key to your server

Copying Your Public Key Using ssh-copy-id

The simplest way to copy your public key to an existing server is to use a utility called ssh-copy-id.

ssh-copy-id -i public_key_filepath username@remote_host
# or use a specific SSH port
ssh-copy-id -i public_key_filepath -p ssh_port username@remote_host

For example

ssh-copy-id -i ~/.ssh/id_ed25519_remote_1.pub -p 38157 root@xxx.xx.xxx.xxx

Copying Your public key using SSH

cat ~/.ssh/id_rsa.pub | ssh username@remote_host "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

• cat ~/.ssh/id_rsa.pub: Output the file.
• mkdir -p ~/.ssh: Creating the ~/.ssh directory if it doesn’t exist.
• cat >> ~/.ssh/authorized_keys: append the standard output of the previous command of the pipeline to the file ~/.ssh/authorized_keys on the remote host.

Configuring SSH

If there are multiple SSH keys on your local system, you need to configure which destination server uses which SSH key. For example, there is an SSH key for GitHub and another SSH key for a remote server.

Creating the SSH configuration file ~/.ssh/config if it doesn’t exist.

vim ~/.ssh/config

Add the config like the folowing content

# GitHub
Host github.com
  User git
  Port 22
  Hostname github.com
  IdentityFile "~/.ssh/{your_private_key}"
  TCPKeepAlive yes
  IdentitiesOnly yes

# Remote server
Host {remote_server_ip_address}
  User {username_for_ssh}
  Port {remote_server_ssh_port}
  IdentityFile "~/.ssh/{your_private_key}"
  TCPKeepAlive yes
  IdentitiesOnly yes

SSH login with the SSH private key

If you have copied your SSH public key to the server, SSH login will automatically use your private key. Otherwise, you will need to enter the password of the remote server’s user to login.

$ ssh username@remote_host
# or use a specific port
$ ssh -p ssh_port username@remote_host

Disabling password authentication on your server

Using password-based authentication exposes your server to brute-force attacks. You can disable password authentication by updating the configuration file /etc/ssh/sshd_config.

Before disabling password authentication, make sure that you either have SSH key-based authentication configured for the root account on this server, or preferably, that you have SSH key-based authentication configured for an account on this server with sudo access.

sudo vim /etc/ssh/sshd_config

Uncomment the following line by removing # at the beginning of the line:

PasswordAuthentication no

Save and close the file when you are finished. To actually implement the changes we just made, you must restart the service.

sudo systemctl restart ssh

References

[1] How To Configure SSH Key-Based Authentication on a Linux Server

[2] Upgrade Your SSH Key to Ed25519

Configuring DNS

Ensure your Linux server’s DNS configuration file /etc/resolv.conf contains some DNS servers.

If there are no DNS servers in the /etc/resolv.conf, you must add some DNS servers to the file.

vim /etc/resolv.conf

Add the following content to the file /etc/resolv.conf

nameserver 192.168.0.1
nameserver 8.8.8.8
nameserver 8.8.4.4

You can try to restart your system networking to check whether the problem of being unable to ping domain names is resolved. See the section “Restart Networking” of this post.

Configuring Default Route Gateway

You need to check your route table and check if the destination host 0.0.0.0 is routed to the default gateway IP (e.g. 192.168.0.1). If not you need to update the gateway IP.

Get Default Gateway IP

$ ip r | grep default

default via 192.168.0.1 dev eth0 proto dhcp metric 100

Some computers might have multiple default gateways. The gateway with lowest Metric is the first to be searched and used as the default gateway.

My server’s default gateway IP is 192.168.0.1.

Check the Route Table

Print the route table:

$ sudo route -n

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1   0.0.0.0         UG    0      0        0 eth0
...

• Destination: The destination network or destination host.
• Gateway: The gateway address or ’*’ if none set.
• Genmask: The netmask for the destination net: 255.255.255.255 for a host destination and 0.0.0.0 for the default route.

What Is The Meaning of 0.0.0.0 In Routing Table?

Each network host has a default route for each network card. This will create a 0.0.0.0 route for such card. The address 0.0.0.0 generally means “any address”. If a packet destination doesn’t match an individual address in the table, it must match a 0.0.0.0 gateway address. In other words, default gateway is always pointed by 0.0.0.0

Update the 0.0.0.0 Route to the Default Gateway

Update the destination host 0.0.0.0 route the the default gateway IP e.g 192.168.0.1.

ip command

You can temporarily update the 0.0.0.0 route gateway by the ip command.

Add the default route:

ip route add default via 192.168.0.1

You can also delete the default route:

ip route delete default

route command

You can temporarily update the 0.0.0.0 route gateway by the route command.

Add the default route:

route add default gw 192.168.0.1

You can also delete the default route:

route del default gw 192.168.0.1

Update configuration file

You can permanently update the 0.0.0.0 route gateway by in system configuration file.

CentOS/RHEL

vim /etc/sysconfig/network

Add the following content to the file /etc/sysconfig/network

NETWORKING=yes
GATEWAY=192.168.0.1

Debian/Ubuntu

vim /etc/network/interfaces

Find network interface and add the following option

... 
gateway 192.168.0.1
...

Restart Networking

After update the gateway configuration file, you need restart the networking.

Restart the networking on CentOS/RHEL

sudo systemctl restart NetworkManager.service
# or
sudo systemctl stop NetworkManager.service
sudo systemctl start NetworkManager.service

Restart the networking on Debian/Ubuntu

sudo /etc/init.d/networking restart

Appendixes

Some public DNS servers

# OpenDNS
208.67.222.222
208.67.220.220

# Cloudflare
1.1.1.1
1.0.0.1

# Google
8.8.8.8
8.8.4.4

# Quad9
9.9.9.9

References

[1] Understanding Routing Table

[2] What Is The Meaning of 0.0.0.0 In Routing Table?

介绍麒麟操作系统

银河麒麟（NeoKylin）是由中国麒麟软件有限公司基于Linux开发的商业操作系统。其社区版为Ubuntu Kylin。中标麒麟（NeoKylin）与银河麒麟同为中国麒麟软件有限公司基于Linux开发的商业操作系统。

银河麒麟高级服务器操作系统V10（Kylin Linux Advanced Server V10 (Tercel)）是针对企业级关键业务，适应虚拟化、云计算、大数据、工业互联网时代对主机系统可靠性、安全性、性能、扩展性和实时性等需求，依据CMMI5级标准研制的提供内生本质安全、云原生支持、自主平台深入优化、高性能、易管理的新一代自主服务器操作系统。银河麒麟系统采用同源构建支持六款自主CPU平台(飞腾、鲲鹏、龙芯、申威、海光、兆芯等国产CPU)，所有组件基于同一套源代码构建。

查看操作系统信息

# 查看 Linux 系统发行版
$ cat /etc/os-release

NAME="Kylin Linux Advanced Server"
VERSION="V10 (Tercel)"
ID="kylin"
VERSION_ID="V10"
PRETTY_NAME="Kylin Linux Advanced Server V10 (Tercel)"
ANSI_COLOR="0;31"

# 查看 CPU 架构
$ lscpu

Architecture:                    aarch64
CPU op-mode(s):                  64-bit
Model name:                      Kunpeng-920
...

Architecture:                    aarch64
CPU op-mode(s):                  64-bit
Byte Order:                      Little Endian
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       HiSilicon
Model:                           0
Model name:                      Kunpeng-920
Stepping:                        0x1
BogoMIPS:                        200.00
NUMA node0 CPU(s):               0-3
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp asimdfhm ssbs

配置 yum 源仓库

使用中标软件的 yum 源仓库

# 备份系统原来的 yum 配置
mv /etc/yum.repos.d/kylin_aarch64.repo /etc/yum.repos.d/kylin_aarch64.repo.bak
# 创建新的 yum 源配置文件
vi /etc/yum.repos.d/kylin_aarch64.repo 

将下面的内容添加到 /etc/yum.repos.d/kylin_aarch64.repo 文件中

[ks10-adv-os]
name=Kylin-Linux-Advanced-Server-os
baseurl=http://update.cs2c.com.cn:8080/NS/V10/V10SP1/os/adv/lic/base/aarch64/
gpgcheck=0
enable=1

baseurl格式为 http://update.cs2c.com.cn:8080/NS/V10/{版本}/os/adv/lic/base/{架构}/

这里的版本选择的是 V10SP1，架构选的是 aarch64（鲲鹏）。

# 清除原有 yum 缓存和生成新的缓存
$ sudo yum clean all && sudo yum makecache
# 测试
$ sudo yum update
$ sudo yum install java-1.8.0-openjdk
Is this ok [y/N]: N

介绍 CTyunOS

天翼云操作系统 CTyunOS 是基于欧拉开源操作系统（openEuler，简称“欧拉”）开发的，欧拉操作系统是华为基于 CentOS 开发的。2019年9月，华为宣布 EulerOS 开源，开源名称为 openEuler（开源欧拉）。EulerOS 支持 AArch64（鲲鹏）处理器、容器虚拟化技术，是一个面向企业级的通用服务器架构平台。华为的鸿蒙是手机手表的操作系统，面向C端。而欧拉是电脑服务器的操作系统，是面向B端。

CTyunOS 操作系统基于 openEuler 20.03 LTS 版本自主研发的。CTyunOS 3 是 CTyunOS 在2023年4月发布的最新版本，上游采用了openEuler社区发布的长期稳定版本 openEuler 22.03 LTS SP1作为基线，针对云计算、云原生场景进行了深度的开发，目前在天翼云上已经可以开通该操作系统的弹性云主机。

查看操作系统信息

# 查看 Linux 系统发行版
$ cat /etc/os-release

NAME="ctyunos"
VERSION="2.0.1"
ID="ctyunos"
VERSION_ID="2.0.1"
PRETTY_NAME="ctyunos 2.0.1"
ANSI_COLOR="0;31"

# 查看 CPU 架构
$ lscpu

Architecture:                    aarch64
CPU op-mode(s):                  64-bit
Model name:                      Kunpeng-920
...

Architecture:                    aarch64
CPU op-mode(s):                  64-bit
Byte Order:                      Little Endian
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       HiSilicon
Model:                           0
Model name:                      Kunpeng-920
Stepping:                        0x1
BogoMIPS:                        200.00
NUMA node0 CPU(s):               0-3
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp asimdfhm ssbs

配置 yum 源仓库

# 备份系统原来的 yum 配置
$ mv ctyunos.repo ctyunos.repo.bak
# 下载华为云的 openeuler yum 源配置文件
$ curl -O https://repo.huaweicloud.com/repository/conf/openeuler_aarch64.repo
# 移动到系统的 yum 源配置目录
$ mv openeuler_aarch64.repo /etc/yum.repos.d
# 清除原有 yum 缓存和生成新的缓存
$ sudo yum clean all && yum makecache
# 测试
$ sudo yum update
$ sudo yum install java-1.8.0-openjdk
Is this ok [y/N]: N

华为云 Linux 软件包仓库配置文件目录：https://repo.huaweicloud.com/repository/conf/

References

[1] Configuring Yum and Yum Repositories - RedHat

[2] 配置OpenEuler的网络yum源

[3] CentOS配置yum仓库的三种方法

File Operations

Archive Files

zip

# Create an achive file
zip -r achive.zip dir_name

# List files of the achive file
unzip -l achive.zip

# Extract files
unzip achive.zip
unzip achive.zip -d /target/filepath

tar

# Create an achive file
tar -czvf achive.tar.gz dir_name
tar -cjvf achive.tar.bz2 dir_name
tar -cvf achive.tar dir_name

# List files of the achive file
tar -tf achive.tar.gz

# Extract files
tar -xvf achive.tar.gz
tar -xvf achive.tar.gz -C /target/filepath

Get information of file

Real path

# Print real path of file
realpath path/to/file

Architecture of software

# Architecture of software
cat `which {your_software}` | file -

# file type
file path/to/file

Files and Directories

Directories

• /lib: It is a place for the essential standard libraries. Think of libraries required for your system to run. If something in /bin or /sbin needs a library that library is likely in /lib.
• /usr
  • /usr/bin: Executables binary files. E.g. java, mvn, git, apt, kill.
  • /usr/local: To keep self-compiled or third-party programs.
  • /usr/sbin: This directory contains programs for administering a system, meant to be run by ‘root’. Like ‘/sbin’, it’s not part of a user’s $PATH. Examples of included binaries here are chroot, useradd, in.tftpd and pppconfig.
  • /usr/share: This directory contains ‘shareable’, architecture-independent files (docs, icons, fonts etc).
  • /usr/lib: the /usr directory in general is as it sounds, a user based directory. Here you will find things used by the users on the system. So if you install an application that needs libraries they might go to /usr/lib. If a binary in /usr/bin or /usr/sbin needs a library it will likely be in /usr/lib.
• /var: variable data files. log files.
  • /var/lib: the /var directory is the writable counterpart to the /usr directory which is often required to be read-only. So /var/lib would have a similar purpose as /usr/lib but with the ability to write to them.

Configuration Files

Bash Configuration Files

Most global config files are located in the /etc directory

Important Global Config Files

Important User-Specific Config Files

System Settings

System Time

Time

# show date time
date

# date time format
date '+%Y-%m-%d'
date '+%Y-%m-%d %H:%M:%S'
date '+%Y-%m-%d_%H-%M-%S'

# update time and date from the internet
timedatectl set-ntp true

Timezone

# list timezones
timedatectl list-timezones

# set timezone
timedatectl set-timezone Asia/Shanghai

# show time settings
timedatectl status

hostname

The hostname is used to distinguish devices within a local network. It’s the machine’s human-friendly name. In addition, computers can be found by others through the hostname, which enables data exchange within a network, for example. Hostnames are used on the internet as part of the fully qualified domain name.

you can configure a computer’s hostname:

# setting
$ hostnamectl set-hostname server1.example.com
# verify the setting
$ less /etc/hostname
# query your computer's hostname
$ hostname

hosts

The /etc/hosts file contains the Internet Protocol (IP) host names and addresses for the local host and other hosts in the Internet network. This file is used to resolve a name into an address (that is, to translate a host name into its Internet address).

sudo vim /etc/hosts

Add Environment Variables

/etc/profile

Add environment variables

cp /etc/profile "/etc/profile.bak.$(date '+%Y-%m-%d_%H-%M-%S')"
echo "export name=value" >> /etc/profile
cat /etc/profile
source /etc/profile

Add to path

cp /etc/profile /etc/profile.bak.$(date '+%Y-%m-%d_%H-%M-%S')
echo 'export PATH=$PATH:/usr/local/mysql/bin' >> /etc/profile
source /etc/profile

System Log

# print OOM killer log
dmesg -T | egrep -i 'killed process'

Upload File

scp

SCP will always overwrite existing files. Thus, in the case of a clean upload SCP should be slightly faster as it doesn’t have to wait for the server on the target system to compare files.

# transfer a file
scp local_file remoteuser@remote_ip_address:/remote_dir
# transfer multiple files
scp local_file1 local_file2 remoteuser@remote_ip_address:/remote_dir

# transfer a directory
scp -r local_dir remoteuser@remote_ip_address:/remote_dir

# transfer a file from remote host to local
scp remoteuser@remote_ip_address:/remote_file local_dir

# Transfer Files Between Two Remote Systems 
scp remoteuser@remote_ip_address:/remote_file remoteuser@remote_ip_address:/remote_file

• -P SSH_port

rsync over ssh

In the case of a synchronization of files that change, like log files or list of source files in a repository, rsync is faster.

Copy a File from a Local Server to a Remote Server with SSH

rsync -avzhe ssh backup.tar.gz root@192.168.0.141:/backups/
# Show Progress While Transferring Data with Rsync
rsync -avzhe ssh --progress backup.tar.gz root@192.168.0.141:/backups/

Copy a File from a Remote Server to a Local Server with SSH

rsync -avzhe ssh root@192.168.0.141:/root/backup.tar.gz /tmp

sftp

sftp [username]@[remote hostname or IP address]

# download file to the local system's Home directory. 
get [path to file]
# change directory
get [path to file] [path to directory]
# change filename
get [path to file] [new file name]

# upload the local system's Home directory's file to the remote server's current directory
put [path to file]
# change directory
put [path to file] [path to directory]
# change filename
put [path to file] [new file name]

Application Data

logging file path: /var/log/{application_name}

upload file path: /data/{application_name}/upload

application build and running file path: /var/java/{application_name}, /var/html/{application_name}

References

• How to configure a hostname on a Linux system
• What is a Hostname?
• hosts File Format for TCP/IP

Reverse Proxy

HTTP

http {
    ...
    server {
        listen 80;
        server_name myserver.com;
        # The default root is /usr/share/nginx/www, /usr/share/nginx/html or /var/www/html
        root /var/www/your_domain/html;

        # Staic files
        location / {
            # redefine the root
            root   /var/www/your_domain/html;
            try_files $uri $uri/index.html =404;
            # if not found redirect to home page $root/index.html
            # try_files $uri $uri/ /index.html;
        }

        # API
        location  /api/ {
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            # The URL of the back-end API has the /api prefix. https://example.com/api/xxx to https://api.example.com/api/xxx.
            proxy_pass http://localhost:8080/api/;
            # The URL of the back-end API does not have the /api prefix. The /api prefix is only used to map to the API interface. https://example.com/api/xxx to https://api.example.com/xxx.
            proxy_pass http://localhost:8080/;
        }

        # Cache js, css, image, etc.
    }
    ...
}

• Response static files: {root path}/requestURI
• Proxy requests: {proxy_pass path}/requestURI

Passing Request Headers

By default, NGINX redefines two header fields in proxied requests, “Host” and “Connection”, and eliminates the header fields whose values are empty strings. “Host” is set to the $proxy_host variable, and “Connection” is set to close.

HTTPS

http {
    # reuse SSL session parameters to avoid SSL handshakes for parallel and subsequent connections.
    # or "ssl_session_cache builtin:1000 shared:SSL:10m;"
    ssl_session_cache   shared:SSL:10m;
    ssl_session_timeout 10m;
    
    server {
        listen 443 ssl;
        server_name myproject.com;
        
        ssl_certificate /etc/ssl/projectName/projectName.com.pem;
        ssl_certificate_key /etc/ssl/projectName/projectName.com.key;
        
        # Additional SSL configuration (if required)
        # enabling keepalive connections to send several requests via one connection and the second is to reuse SSL session parameters to avoid SSL handshakes for parallel and subsequent connections.
        keepalive_timeout   70;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
        ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
        ssl_prefer_server_ciphers on;

        # same with HTTP
        location / {
            ...
        }
    }
    ...
}

HTTP to HTTPS

http {
    ...
    server {
        listen 80;
        server_name myserver.com;
        return 301 https://myserver.com$request_uri;
    }
    ...
}

Optimization

Enable HTTP2

The ngx_http_v2_module module (1.9.5) provides support for HTTP/2. This module is not built by default, it should be enabled with the --with-http_v2_module configuration parameter.

To enable HTTP/2, you must first enable SSL/TLS on your website. HTTP/2 requires the use of SSL/TLS encryption, which provides a secure connection between the web server and the client’s browser.

http {
    server {
        # enable http2
        listen 443 ssl http2;
        
        ssl_certificate server.crt;
        ssl_certificate_key server.key;
    }
}

Cache static files

When you build static assets with versioning/hashing mechanisms, adding a version/hash to the filename or query string is a good way to manage caching. In such a case, you can add a long max-age value and immutable because the content will never change.

http {
    server {
        ...
        location /static {
            root   /var/www/your_domain/staic;
            # To disable access log off for not hitting the I/O limit
            access_log off;
            # or "expires max";
            expires 7d;
            # build static assets with versioning/hashing mechanisms
            add_header Cache-Control "public, immutable";
            # revalidate
            # add_header Cache-Control "public, must-revalidate, proxy-revalidate";
        }
        ...
    }
}

http {
    server {
        ...
        location ~* \.(js|css|png|jpg|jpeg|gif|svg|ico)$ {
            root   /var/www/your_domain/staic;
            # To disable access log off for not hitting the I/O limit
            access_log off;
            # or "expires max";
            expires 7d;
            # build static assets with versioning/hashing mechanisms
            add_header Cache-Control "public, immutable";
            # revalidate
            # add_header Cache-Control "public, must-revalidate, proxy-revalidate";
        }
        ...
    }
}

After setting cache, the following headers will be present in the response headers:

Cache-Control: max-age=604800, public, must-revalidate, proxy-revalidate

Warning: Cache-Control "public, immutable" for JS, CSS may cause program exceptions after update program.

Compression

Enable gzip

http {
    ...
    server {
        ...
        # gzip can be set in `http {}` or `server {}`
        # --with-http_gzip_static_module
        gzip on;
        # By default, NGINX compresses responses only with MIME type text/html. To compress responses with other MIME types, include the gzip_types directive and list the additional types.
        gzip_types text/css text/xml application/javascript;
        # To specify the minimum length of the response to compress, use the gzip_min_length directive. The default is 20 bytes
        gzip_min_length 200;
        # Sets the number and size of buffers used to compress a response.
        gzip_buffers 32 4k;
        # Sets a gzip compression level of a response. Acceptable values are in the range from 1 to 9.
        gzip_comp_level 6;
        gzip_vary on;
        ...
    }
}

After enable gzip, the following headers will be present in the response headers:

content-encoding: gzip

Settings

Timeout

http {
    ...
    # proxy_connect_timeout default 60s
    proxy_connect_timeout 180s;
    # proxy_send_timeout default 60s
    proxy_send_timeout 180s;
    # proxy_read_timeout default 60s
    proxy_read_timeout 180s;
    ...
}

Upload File Size

http {
    ...
    # client_max_body_size default 1M
    client_max_body_size 100M;
    ...
}

Enable CORS for API

Enable CORS for specified sites

http {
    ...

    map "$http_origin" $cors {
      default '';
      "~^https?://localhost(:\d+)?$" "$http_origin";
      "~^https?://10.0.0\.\d{1,2}(:\d+)?$" "$http_origin";
      "~^https?://example\.com$" "$http_origin";
    }

    server {
        # API
        location  /api/ {
            ...
            
            # Attach CORS headers only if it's a valid origin ($cors should not be empty)
            if ($cors != "") {
                # using $cors for specified sites or using $http_origin for any sites.
                proxy_hide_header 'Access-Control-Allow-Origin';
                add_header 'Access-Control-Allow-Origin' '$cors' always;
                proxy_hide_header 'Access-Control-Allow-Credentials';
                add_header 'Access-Control-Allow-Credentials' true always;
                proxy_hide_header 'Access-Control-Allow-Methods';
                add_header 'Access-Control-Allow-Methods' 'POST, GET, DELETE, PUT, PATCH' always;
                proxy_hide_header 'Access-Control-Allow-Headers';
                add_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With' always;
            }
            # Check if it's a preflight request and "cache" it for 20 days
            if ($request_method = 'OPTIONS') {
                add_header 'Access-Control-Allow-Origin' '$cors' always;
                add_header 'Access-Control-Allow-Credentials' true always;
                add_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With' always;
                add_header 'Access-Control-Max-Age' 1728000;
                add_header 'Content-Type' 'text/plain charset=UTF-8';
                add_header 'Content-Length' 0;
                return 204;
            }
            
            ...
        }
    }
}

Enable CORS for all sites

http {
    ...

    server {
        # API
        location  /api/ {
            ...
            
            proxy_hide_header 'Access-Control-Allow-Origin';
            add_header 'Access-Control-Allow-Origin' '$http_origin' always;
            proxy_hide_header 'Access-Control-Allow-Credentials';
            add_header 'Access-Control-Allow-Credentials' true always;
            proxy_hide_header 'Access-Control-Allow-Methods';
            add_header 'Access-Control-Allow-Methods' 'POST, GET, DELETE, PUT, PATCH' always;
            proxy_hide_header 'Access-Control-Allow-Headers';
            add_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With' always;

            # Check if it's a preflight request and "cache" it for 20 days
            if ($request_method = 'OPTIONS') {
                add_header 'Access-Control-Allow-Origin' '$http_origin' always;
                add_header 'Access-Control-Allow-Credentials' true always;
                add_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With' always;
                add_header 'Access-Control-Max-Age' 1728000;
                add_header 'Content-Type' 'text/plain charset=UTF-8';
                add_header 'Content-Length' 0;
                return 204;
            }
            
            ...
        }
    }
}

Load Balancing

http {
    ...
    upstream backend-server {
        server xxx.xxx.xxx.xxx:8080 max_fails=1 fail_timeout=300s;
        server xxx.xxx.xxx.xxx:8080 max_fails=1 fail_timeout=300s;
        ...
    }

    server {
        ...
        location /api/ {
            proxy_set_header    X-Real-IP        $remote_addr;
            proxy_set_header    X-Forwarded-For  $proxy_add_x_forwarded_for;
            proxy_pass http://backend-server/;
        }
    }
}

Test the Nginx Configuration is Updated

Adding the following config to the Nginx configuration file. You can verify if the configuration is updated by updating the return status code (e.g. 403 Forbidden, 406 Not Acceptable, 423 Locked) of the /test location and visiting the test URL http://yourDomain/testConfig.

location /testConfig {
    # 403 Forbidden, 406 Not Acceptable, 423 Locked
    return 403;
}

Appendixes

Embedded Variables

• $proxy_host: name and port of a proxied server as specified in the proxy_pass directive;
• $proxy_add_x_forwarded_for: the “X-Forwarded-For” client request header field with the $remote_addr variable appended to it, separated by a comma. If the “X-Forwarded-For” field is not present in the client request header, the $proxy_add_x_forwarded_for variable is equal to the $remote_addr variable.
• $host: In this order of precedence: host name from the request line, or host name from the “Host” request header field, or the server name matching a request.
• $remote_addr: Client address

Build Nginx From Source

# Download Nginx source code
wget http://nginx.org/download/nginx-{latest-stable-version}.tar.gz

You can know the latest version of Nginx by visiting the Nginx download page.

tar -zxvf nginx-{latest-stable-version}.tar.gz

cd nginx-{latest-stable-version}

# Configuring Nginx
./configure \
--with-pcre \
--with-http_ssl_module \
--with-http_image_filter_module=dynamic \
--modules-path=/etc/nginx/modules \
--with-http_v2_module \
--with-stream=dynamic \
--with-http_addition_module \
--with-http_mp4_module \
--with-http_gzip_static_module

More configuration

Common errors when running ./configure

1. ./configure: error: the HTTP rewrite module requires the PCRE library.

Solution

sudo apt update && apt upgrade -y && apt autoremove && apt autoclean
apt-get install libpcre3 libpcre3-dev
# or
sudo yum update -y && yum upgrade && yum autoremove
yum install libpcre3 libpcre3-dev
yum -y install pcre-devel openssl openssl-devel

2. ./configure: error: the HTTP image filter module requires the GD library. You can either do not enable the module or install the libraries.

Solution

sudo apt update && apt upgrade -y && apt autoremove && apt autoclean
apt-get install gd-devel
# or
sudo yum update -y && yum upgrade && yum autoremove
yum install gd-devel -y

3. ./configure: error: C compiler cc is not found

Solution

sudo apt update && apt upgrade -y && apt autoremove && apt autoclean
sudo apt-get install linux-kernel-headers build-essential -y
# or
sudo yum update -y && yum upgrade && yum autoremove
sudo yum install gcc -y

Successful output of configure

Configuration summary
  + using system PCRE2 library
  + using system OpenSSL library
  + using system zlib library

  nginx path prefix: "/usr/local/nginx"
  nginx binary file: "/usr/local/nginx/sbin/nginx"
  nginx modules path: "/etc/nginx/modules"
  nginx configuration prefix: "/usr/local/nginx/conf"
  nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
  nginx pid file: "/usr/local/nginx/logs/nginx.pid"
  nginx error log file: "/usr/local/nginx/logs/error.log"
  nginx http access log file: "/usr/local/nginx/logs/access.log"
  nginx http client request body temporary files: "client_body_temp"
  nginx http proxy temporary files: "proxy_temp"
  nginx http fastcgi temporary files: "fastcgi_temp"
  nginx http uwsgi temporary files: "uwsgi_temp"
  nginx http scgi temporary files: "scgi_temp"

You can add the following parameters to specify paths:

--prefix=/var/www/html \
--sbin-path=/usr/sbin/nginx \
--modules-path=/etc/nginx/modules \
--conf-path=/etc/nginx/nginx.conf \
--pid-path=/var/run/nginx.pid \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--lock-path=/var/lock/nginx.lock \

# Build nginx
$ make
$ sudo make install

Successful output of build

make -f objs/Makefile install
make[1]: 进入目录“/root/yuqing/nginx-1.26.1”
...
# make[1]: 离开目录“/root/yuqing/nginx-1.26.1”

# Start nginx
$ cd /usr/local/nginx/sbin
$ ./nginx -V
$ ./nginx
# Verify
$ curl http://localhost

Rebuild Nginx source

# Just remove the nginx binary file. Or completely remove nginx `sudo apt-get purge nginx` or `yum remove package`
cd /usr/local/nginx/sbin
mv nginx nginx.bak

# configure
tar -zxvf nginx-{latest-stable-version}.tar.gz
cd nginx-{latest-stable-version}
# Configuring Nginx
./configure ...

# Build and install nginx
$ make
$ sudo make install

References

[1] Configuring HTTPS servers - Nginx

[2] Alphabetical index of variables - Nginx

[3] Serving Static Content - Nginx

[4] NGINX Reverse Proxy - Nginx