Troubleshooting RLC¶

This guide covers common issues and solutions when working with Rocky Linux from CIQ (RLC).

Repository and Access Issues¶

Depot Access Problems¶

Problem: Cannot access CIQ Depot repositories

Solutions: 1. Verify credentials:

curl -u USERNAME:TOKEN https://depot.ciq.com/test

Check network connectivity:

ping depot.ciq.com
curl -I https://depot.ciq.com

Verify repository configuration:

dnf repolist enabled
cat /etc/yum.repos.d/rlc.repo

Clear repository cache:
```
sudo dnf clean all
sudo dnf makecache
```

Package Installation Failures¶

Problem: DNF package installation fails

Solutions: 1. Update repository metadata:

sudo dnf clean metadata
sudo dnf makecache

Check available space:
```
df -h /var/cache/dnf
df -h /tmp
```

Verify GPG keys:

rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-*
dnf check-update

Debug package issues:

dnf --debuglevel=10 install package-name

Authentication Issues¶

Problem: Repository authentication fails

Solutions: 1. Check credentials file:

cat ~/.netrc
# Should contain: machine depot.ciq.com login USERNAME password TOKEN

Verify permissions:
```
chmod 600 ~/.netrc
ls -la ~/.netrc
```

Test authentication:

curl -n https://depot.ciq.com/private/test

Installation Issues¶

Boot Problems¶

Problem: System fails to boot after installation

Solutions: 1. Check boot order in BIOS/UEFI 2. Boot from rescue media:

# Boot from installation media
# Select "Troubleshooting" → "Rescue"

Repair GRUB:

grub2-install /dev/sda
grub2-mkconfig -o /boot/grub2/grub.cfg

Check filesystem errors:
```
fsck /dev/sda1
```

Network Configuration Issues¶

Problem: Network not working after installation

Solutions: 1. Check network interface status:

ip addr show
nmcli device status

Restart NetworkManager:
```
sudo systemctl restart NetworkManager
```

Configure network manually:

sudo nmcli con mod "System eth0" ipv4.addresses 192.168.1.100/24
sudo nmcli con mod "System eth0" ipv4.gateway 192.168.1.1
sudo nmcli con mod "System eth0" ipv4.dns 8.8.8.8
sudo nmcli con mod "System eth0" ipv4.method manual
sudo nmcli con up "System eth0"

System Performance Issues¶

High CPU Usage¶

Problem: Unexpectedly high CPU usage

Solutions: 1. Identify high CPU processes:

top
htop
ps aux --sort=-%cpu | head -10

Check for runaway processes:
```
ps aux | grep -E "(100|99)\.[0-9]"
```
Monitor system load:
```
uptime
sar -u 1 10
```

Check for hardware issues:

dmesg | grep -i error
journalctl -p err --since "1 hour ago"

Memory Issues¶

Problem: High memory usage or out of memory errors

Solutions: 1. Check memory usage:

free -h
cat /proc/meminfo
sudo smem -t

Identify memory-hungry processes:
```
ps aux --sort=-%mem | head -10
```
Check for memory leaks:
```
valgrind --leak-check=full command
```

Configure swap if needed:

sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Disk Space Issues¶

Problem: Running out of disk space

Solutions: 1. Check disk usage:

df -h
du -sh /*

Find large files:
```
find / -type f -size +100M 2>/dev/null
```
Clean package cache:
```
sudo dnf clean all
```

Clean logs:

sudo journalctl --vacuum-time=7d
sudo logrotate -f /etc/logrotate.conf

Service and Application Issues¶

Service Startup Failures¶

Problem: System services fail to start

Solutions: 1. Check service status:

sudo systemctl status service-name
sudo systemctl --failed

Review service logs:

sudo journalctl -u service-name
sudo journalctl -f -u service-name

Check dependencies:

systemctl list-dependencies service-name

Restart service:

sudo systemctl restart service-name
sudo systemctl enable service-name

Database Connection Issues¶

Problem: Cannot connect to database

Solutions: 1. Check database service:

sudo systemctl status mariadb
sudo systemctl status postgresql

Verify database configuration:

sudo mysql -u root -p
sudo -u postgres psql

Check network connectivity:

telnet database-server 3306
netstat -tulpn | grep :3306

Review database logs:

sudo tail -f /var/log/mysql/error.log
sudo tail -f /var/log/postgresql/postgresql.log

Web Server Issues¶

Problem: Web server not responding

Solutions: 1. Check web server status:

sudo systemctl status httpd
sudo systemctl status nginx

Test configuration:
```
sudo httpd -t
sudo nginx -t
```

Check port availability:

netstat -tulpn | grep :80
netstat -tulpn | grep :443

Review error logs:

sudo tail -f /var/log/httpd/error_log
sudo tail -f /var/log/nginx/error.log

Network Issues¶

DNS Resolution Problems¶

Problem: DNS not resolving properly

Solutions: 1. Check DNS configuration:

cat /etc/resolv.conf
systemd-resolve --status

Test DNS resolution:
```
nslookup google.com
dig google.com
```

Flush DNS cache:

sudo systemctl restart systemd-resolved

Configure alternative DNS:

echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf

Firewall Issues¶

Problem: Network connectivity blocked by firewall

Solutions: 1. Check firewall status:

sudo firewall-cmd --state
sudo firewall-cmd --list-all

Temporarily disable firewall:
```
sudo systemctl stop firewalld
```

Add firewall rules:

sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-port=8080/tcp
sudo firewall-cmd --reload

Check iptables rules:
```
sudo iptables -L -n
```

Security Issues¶

SELinux Problems¶

Problem: SELinux blocking applications

Solutions: 1. Check SELinux status:

getenforce
sestatus

Review SELinux denials:

sudo sealert -a /var/log/audit/audit.log
sudo ausearch -m avc -ts recent

Temporarily set permissive mode:
```
sudo setenforce 0
```

Create SELinux policy:

sudo audit2allow -M mypolicy < /var/log/audit/audit.log
sudo semodule -i mypolicy.pp

Permission Issues¶

Problem: File permission problems

Solutions: 1. Check file permissions:

ls -la /path/to/file

Fix ownership:
```
sudo chown user:group /path/to/file
```

Fix permissions:

sudo chmod 644 /path/to/file
sudo chmod 755 /path/to/directory

Restore SELinux contexts:
```
sudo restorecon -R /path/to/directory
```

Update and Upgrade Issues¶

Failed Updates¶

Problem: Package updates fail

Solutions: 1. Check for conflicts:

sudo dnf check
sudo dnf distro-sync --allowerasing

Clear update cache:
```
sudo dnf clean all
sudo dnf makecache
```
Update specific packages:
```
sudo dnf update package-name
```

Skip problematic packages:

sudo dnf update --exclude=problematic-package

Kernel Update Issues¶

Problem: System won't boot after kernel update

Solutions: 1. Boot from older kernel: - Select older kernel from GRUB menu

Remove problematic kernel:
```
sudo dnf remove kernel-5.x.x
```

Reinstall kernel:

sudo dnf reinstall kernel
sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Log Analysis and Debugging¶

System Log Analysis¶

# Check system logs for errors
sudo journalctl -p err --since "24 hours ago"
sudo journalctl -f

# Check specific service logs
sudo journalctl -u service-name --since "1 hour ago"

# Check boot logs
sudo journalctl -b

# Check kernel messages
dmesg | tail -20

Performance Debugging¶

# Monitor system performance
sar -u 1 10    # CPU usage
sar -r 1 10    # Memory usage
sar -d 1 10    # Disk I/O
sar -n DEV 1 10  # Network usage

# Check system load
uptime
top
htop

Getting Help¶

Information Collection¶

Before contacting support, collect:

# System information
uname -a
cat /etc/os-release
hostnamectl

# Hardware information
lscpu
free -h
lsblk

# Network information
ip addr show
ip route show

# Service status
systemctl --failed
systemctl list-unit-files --state=enabled

# Recent logs
sudo journalctl --since "24 hours ago" -p err

Support Channels¶

CIQ Professional Support: Enterprise customers with support contracts
Documentation: Latest guides and troubleshooting resources
Community Forums: User community discussions
Knowledge Base: Searchable database of solutions

Emergency Recovery¶

If system is unbootable:

Boot from rescue media
Mount root filesystem:
```
mount /dev/sdXY /mnt
chroot /mnt
```
Fix issues and reboot
Contact support if needed

For additional assistance, contact CIQ support or consult the latest documentation.