1 Commits

Author SHA1 Message Date
ShadowNinja
46ff8cdaf0 Add support for persistent storage using MongoDB 2016-01-31 14:17:06 -05:00
12 changed files with 423 additions and 955 deletions

View File

@@ -1,8 +0,0 @@
root = true
[*]
charset = utf-8
indent_style = tab
indent_size = 4
trim_trailing_whitespace = true
insert_final_newline = true

View File

@@ -1,28 +0,0 @@
name: lint
on:
push:
paths:
- '**.py'
- 'requirements.txt'
pull_request:
paths:
- '**.py'
- 'requirements.txt'
jobs:
pylint:
runs-on: ubuntu-latest
container:
image: python:3.11-slim
steps:
- uses: actions/checkout@v3
- name: Install deps
run: |
pip install -r requirements.txt
pip install pylint
- name: Lint
run: |
pylint -E --fail-on=E server.py

9
.gitignore vendored
View File

@@ -1,8 +1,7 @@
*~
node_modules
__pycache__
/store.json
/static/list.json
/static/servers.js
/config.py
/*.mmdb
static/list.json
static/servers.js
config.py

168
README.md
View File

@@ -1,4 +1,4 @@
Luanti server list
Minetest server list
====================
Setting up the webpage
@@ -9,141 +9,101 @@ the server list webpage template.
First install node.js, e.g.:
```sh
apt-get install nodejs
# OR:
yum install nodejs
```
# apt-get install nodejs
# # OR:
# pacman -S nodejs
# # OR:
# emerge nodejs
Then install doT.js and its dependencies:
```sh
npm install dot "commander@11.1.0" mkdirp
```
$ cd ~
$ npm install dot commander mkdirp
And finally compile the template:
```sh
cd static
../node_modules/dot/bin/dot-packer -s .
```
$ cd static
$ ~/node_modules/dot/bin/dot-packer -s . -d .
You can now serve the webpage by copying the files in `static/` to your web root, or by [starting the server list](#setting-up-the-server).
You can now serve the webpage by copying the files in static/ to your web root, or by [starting the master server](#setting-up-the-server).
Embedding the server list in a page
-----------------------------------
```html
<head>
...
<script>
var master = {
root: 'https://servers.luanti.org/',
limit: 10,
clients_min: 1,
no_flags: true,
no_ping: true,
no_uptime: true
};
</script>
...
</head>
<body>
...
<div id="server_list"></div>
...
<script defer src="https://servers.luanti.org/list.js"></script>
</body>
```
<head>
...
<script>
var master = {
root: 'http://servers.minetest.net/',
limit: 10,
clients_min: 1,
no_flags: 1,
no_ping: 1,
no_uptime: 1
};
</script>
...
</head>
<body>
...
<div id="server_list"></div>
...
</body>
<script src="list.js"></script>
Setting up the server
---------------------
1. Install Python 3 and pip:
```sh
apt-get install python3 python3-pip
# OR:
yum install python3 python3-pip
```
# pacman -S python python-pip
# # OR:
# apt-get install python3 python3-pip
2. Install required Python packages:
pip3 install -r requirements.txt
# # You might have to use pip3 if your system defaults to Python 2
# pip install -r requirements.txt
3. If using in production, install uwsgi and its python plugin:
3. If using in production, install uwsgi and it's python plugin:
```sh
apt-get install uwsgi-plugin-python3
# OR:
yum install uwsgi uwsgi-plugin-python3
```
# pacman -S uwsgi uwsgi-plugin-python
# # OR:
# apt-get install uwsgi uwsgi-plugin-python
# # OR:
# pip install uwsgi
4. Configure the server by adding options to `config.py`.
See `config-example.py` for defaults.
4. Install, start, and enable MongoDB on boot:
5. Start the server:
# pacman -S mongodb && systemctl enable mongodb --now
```sh
./server.py
# Or for production:
uwsgi -s /run/serverlist.sock --plugins python3 -w server:app -T --threads 2
# then configure according to https://flask.palletsprojects.com/en/stable/deploying/uwsgi/
```
5. Configure the server by adding options to `config.py`.
See `config-example.py` for defaults.
6. Start the server:
$ ./server.py
$ # Or for production:
$ uwsgi -s /tmp/minetest-master.sock --plugin python -w server:app --enable-threads
$ # Then configure according to http://flask.pocoo.org/docs/deploying/uwsgi/
7. (optional) Configure the proxy server, if any. You should make the server
load static files directly from the static directory. Also, `/list`
should be served from `list.json`. Example for nginx:
load static files directly from the static directory. Also, `/list`
should be served from `list.json`. Example for nginx:
```sh
root /path/to/server/static;
rewrite ^/$ /index.html break;
rewrite ^/list$ /list.json break;
location = /list.json { expires 20s; }
try_files $uri @uwsgi;
location @uwsgi {
include uwsgi_params;
uwsgi_pass unix:/run/serverlist.sock;
}
```
Setting up the server (Apache version)
--------------------------------------
If you wish to use Apache to host the server list, do steps 1-2, 4, above.
Additionally install/enable mod_wsgi and an Apache site config like the following:
```sh
# This config assumes you have the server list at DocumentRoot.
# Visitors to the server list in this config would visit http://local.server/ and
# apache would serve up the output from server.py.
# Where are the serverlist files located?
DocumentRoot /var/games/luanti/serverlist
# Serve up server.py at the root of the URL.
WSGIScriptAlias / /var/games/luanti/serverlist/server.py
# The name of the function that we call when we invoke server.py
WSGICallableObject app
# These options are necessary to enable Daemon mode. Without this, you'll have strange behavior
# with servers dropping off your list! You can tweak threads as needed. See mod_wsgi documentation.
WSGIProcessGroup luanti-serverlist
WSGIDaemonProcess luanti-serverlist threads=2
<Directory /var/games/luanti/serverlist>
Require all granted
</Directory>
```
root /path/to/server/static;
rewrite ^/list$ /list.json;
try_files $uri @uwsgi;
location @uwsgi {
uwsgi_pass ...;
}
License
-------
The Luanti server list code is licensed under the GNU Lesser General Public
The Minetest master server is licensed under the GNU Lesser General Public
License version 2.1 or later (LGPLv2.1+). A LICENSE.txt file should have been
supplied with your copy of this software containing a copy of the license.

View File

@@ -1,3 +1,5 @@
from datetime import timedelta
# Enables detailed tracebacks and an interactive Python console on errors.
# Never use in production!
DEBUG = False
@@ -7,24 +9,23 @@ HOST = "127.0.0.1"
# Port for development server to listen on
PORT = 5000
# Amount of time, is seconds, after which servers are removed from the list
# if they haven't updated their listings. Note: By default Luanti servers
# only announce once every 5 minutes, so this should be more than 300.
PURGE_TIME = 350
# Amount of time after which servers are removed from the list if they haven't
# updated their listings. Note: By default Minetest servers only announce
# once every 5 minutes, so this should be more than that.
UPDATE_TIME = timedelta(minutes=6)
# List of banned IP addresses for announce
# e.g. ['2620:101::44']
BANNED_IPS = []
# List of banned servers as host/port pairs
# e.g. ['1.2.3.4/30000', 'lowercase.hostname', 'lowercase.hostname/30001']
BANNED_SERVERS = []
# Amount of time after which servers are removed from the database if they
# haven't updated their listings.
PURGE_TIME = timedelta(days=30)
# Creates server entries if a server sends an 'update' and there is no entry yet.
# This should only be used to populate the server list after list.json was deleted.
# This WILL cause problems such as mapgen, mods and privilege information missing from the list
ALLOW_UPDATE_WITHOUT_OLD = False
# Reject servers with private addresses and domain names.
# Enable this if you are running a list on the public internet.
REJECT_PRIVATE_ADDRESSES = False
# Number of days' data to factor into popularity calculation
POP_DAYS = 3
# Address of the MongoDB server. You can use domain sockets on unix.
MONGO_URI = "mongodb://localhost/minetest-master"

View File

@@ -1,2 +1,4 @@
Flask>=2.0.0
maxminddb>=2.0.0
APScheduler>=3
Flask>=0.10
Flask-PyMongo>=0.3

847
server.py

File diff suppressed because it is too large Load Diff

View File

@@ -2,36 +2,10 @@
<html>
<head>
<meta charset="utf-8">
<title>Luanti server list</title>
<link rel="stylesheet" href="modern-normalize.min.css">
<style>
body {
margin: .5em;
}
a {
color: #336B87;
}
a:visited {
color: #336BA1;
}
hr {
border: 0;
border-top: 3px solid #53ac56;
}
@media only screen and (max-width: 1024px) {
#server_list table .version, #server_list table .flags, #server_list table .uptime {
display: none;
}
}
</style>
<script>
var master = {show_proto_select: true};
</script>
<title>Minetest server list</title>
</head>
<body>
<span class="h"><strong>Luanti server list</strong> | <a href="https://www.luanti.org/get-involved/#reporting-issues">Contact</a> | <a href="https://www.luanti.org/app-privacy-policy/">Privacy</a></span>
<hr />
<div id="server_list"></div>
<script src="list.js"></script>
</body>
</html>
<script src="list.js"></script>

View File

@@ -1,146 +1,101 @@
var master;
if (!master)
master = {};
if (!master.root)
master.root = window.location.href;
if (!master.list)
master.list = "list";
if (!master.list_root)
master.list_root = master.root;
if (!master.list_url)
master.list_url = master.list_root + master.list;
master.cached_json = null;
// Utility functions used by the templating code
if (!master) master = {};
if (typeof(master.root) == 'undefined') master.root = window.location.href;
if (!master.output) master.output = '#server_list';
if (!master.list) master.list = "list";
if (!master.list_root) master.list_root = master.root;
if (!master.list_url) master.list_url = master.list_root + master.list;
function humanTime(seconds) {
if (typeof(seconds) != "number")
return '?';
if (typeof(seconds) != "number") return '?';
var conv = {
y: 31536000,
d: 86400,
h: 3600,
m: 60
};
}
for (var i in conv) {
if (seconds >= conv[i]) {
return (seconds / conv[i]).toFixed(i=='y'?1:0) + i;
return (seconds / conv[i]).toFixed(1) + i;
}
}
return seconds + 's';
}
function escapeHTML(str) {
if (!str)
return str;
if(!str) return str;
return str.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;').replace(/"/g, '&quot;');
}
function addressString(server) {
var addrStr = server.address;
if (addrStr.indexOf(':') != -1)
addrStr = '[' + addrStr + ']';
var isIPv6 = server.address.indexOf(":") != -1;
var addrStr = (isIPv6 ? '[' : '') +
escapeHTML(server.address) +
(isIPv6 ? ']' : '');
var shortStr = addrStr;
addrStr += ':' + server.port;
var str = '<span'
if (shortStr.length > 26) {
shortStr = shortStr.substring(0, 25) + "\u2026";
str += ' title="' + escapeHTML(addrStr) + '"'
if (shortStr.length > 25) {
shortStr = shortStr.substr(0, 23) + "&hellip;";
str += ' class="mts_tooltip" title="' + addrStr + '"'
}
if (server.port != 30000)
shortStr += ':' + server.port;
return str + '>' + escapeHTML(shortStr) + '</span>';
return str + '>' + shortStr + '</span>';
}
function tooltipString(str) {
function tooltipString(str, maxLen) {
str = escapeHTML(str);
return '<span title="' + str + '">' + str + '</div>';
var shortStr = str;
var ret = '<span';
if (shortStr.length > maxLen) {
shortStr = shortStr.substr(0, maxLen - 2) + "&hellip;";
ret += ' class="mts_tooltip" title="' + str + '"';
}
return ret + '>' + shortStr + '</span>';
}
function hoverList(name, list) {
if (!list || list.length == 0)
return '';
if (!list || list.length == 0) return '';
var str = '<div class="mts_hover_list">'
str += '<b>' + escapeHTML(name) + '</b> (' + list.length + ')<br />';
str += name + ' (' + list.length + ')<br />';
for (var i in list) {
str += escapeHTML(list[i]) + '<br />';
}
return str + '</div>';
}
function hoverString(name, str) {
if (!str)
return '';
if (typeof(str) != 'string')
str = str.toString();
return '<div class="mts_hover_list">'
+ '<b>' + escapeHTML(name) + '</b>:<br />'
+ escapeHTML(str) + '<br />'
function hoverString(name, string) {
if (!string) return '';
return '<div class="mts_hover_list">'
+ name + ':<br />'
+ escapeHTML(string) + '<br />'
+ '</div>';
}
function constantWidth(str, width) {
if (typeof(str) != 'string')
str = str.toString();
return '<span class="mts_cwidth" style="width:' + width + 'em;">' + escapeHTML(str) + '</span>';
function draw(json) {
var html = window.render.servers(json);
jQuery(master.output).html(html);
}
// Code that fetches & displays the actual list
function get() {
jQuery.getJSON(master.list_url, draw);
}
master.draw = function(json) {
if (json == null)
return;
// pre-filter by chosen protocol range
var tmp = master.proto_range ? JSON.parse(master.proto_range) : null;
if (tmp) {
json = {
list: json.list.filter(function(server) {
return !(tmp[0] > server.proto_max || tmp[1] < server.proto_min);
}),
total: {clients: 0},
total_max: {clients: "?", servers: "?"}
};
json.list.forEach(function(server) { json.total.clients += server.clients; });
json.total.servers = json.list.length;
function loaded(){
if (!master.no_refresh) {
setInterval(get, 60 * 1000);
}
var html = window.render.servers(json);
jQuery('#server_list').html(html);
jQuery('.proto_select', '#server_list').on('change', function(e) {
master.proto_range = e.target.value;
master.draw(master.cached_json); // re-render
});
};
master.get = function() {
jQuery.getJSON(master.list_url, function(json) {
master.cached_json = json;
master.draw(json);
});
};
master.loaded = function() {
if (!master.no_refresh)
setInterval(master.get, 60 * 1000);
master.get();
};
master.showAll = function() {
delete master.min_clients;
delete master.limit;
master.get();
};
get();
}
// https://github.com/pyrsmk/toast
this.toast=function(){var e=document,t=e.getElementsByTagName("head")[0],n=this.setTimeout,r="createElement",i="appendChild",s="addEventListener",o="onreadystatechange",u="styleSheet",a=10,f=0,l=function(){--f},c,h=function(e,r,i,s){if(!t)n(function(){h(e)},a);else if(e.length){c=-1;while(i=e[++c]){if((s=typeof i)=="function"){r=function(){return i(),!0};break}if(s=="string")p(i);else if(i.pop){p(i[0]),r=i[1];break}}d(r,Array.prototype.slice.call(e,c+1))}},p=function(n,s){++f,/\.css$/.test(n)?(s=e[r]("link"),s.rel=u,s.href=n,t[i](s),v(s)):(s=e[r]("script"),s.src=n,t[i](s),s[o]===null?s[o]=m:s.onload=l)},d=function(e,t){if(!f)if(!e||e()){h(t);return}n(function(){d(e,t)},a)},v=function(e){if(e.sheet||e[u]){l();return}n(function(){v(e)},a)},m=function(){/ded|co/.test(this.readyState)&&l()};h(arguments)};
toast(master.root + 'style.css', master.root + 'servers.js', function() {
if (typeof(jQuery) != 'undefined')
return master.loaded();
return loaded();
else
toast('//ajax.googleapis.com/ajax/libs/jquery/3.7.1/jquery.min.js', master.loaded);
toast('//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js', loaded);
});

View File

@@ -1,9 +0,0 @@
/**
* Minified by jsDelivr using clean-css v5.3.2.
* Original file: /npm/modern-normalize@3.0.1/modern-normalize.css
*
* Do NOT use SRI with dynamically generated files! More information: https://www.jsdelivr.com/using-sri-with-dynamic-files
*/
/*! modern-normalize v3.0.1 | MIT License | https://github.com/sindresorhus/modern-normalize */
*,::after,::before{box-sizing:border-box}html{font-family:system-ui,'Segoe UI',Roboto,Helvetica,Arial,sans-serif,'Apple Color Emoji','Segoe UI Emoji';line-height:1.15;-webkit-text-size-adjust:100%;tab-size:4}body{margin:0}b,strong{font-weight:bolder}code,kbd,pre,samp{font-family:ui-monospace,SFMono-Regular,Consolas,'Liberation Mono',Menlo,monospace;font-size:1em}small{font-size:80%}sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}table{border-color:currentcolor}button,input,optgroup,select,textarea{font-family:inherit;font-size:100%;line-height:1.15;margin:0}[type=button],[type=reset],[type=submit],button{-webkit-appearance:button}legend{padding:0}progress{vertical-align:baseline}::-webkit-inner-spin-button,::-webkit-outer-spin-button{height:auto}[type=search]{-webkit-appearance:textfield;outline-offset:-2px}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-file-upload-button{-webkit-appearance:button;font:inherit}summary{display:list-item}
/*# sourceMappingURL=/sm/d2d8cd206fb9f42f071e97460f3ad9c875edb5e7a4b10f900a83cdf8401c53a9.map */

View File

@@ -1,26 +1,18 @@
{{? !master.no_total}}
<div>
<span class="header_total">
Players: {{=it.total.clients}}/{{=it.total_max.clients}}&nbsp;
Servers: {{=it.total.servers}}/{{=it.total_max.servers}}
</span>
{{? master.show_proto_select}}
, Protocol: <select class="proto_select">
<option value="">All</option>
<option value="[11,32]" {{? master.proto_range=='[11,32]'}}selected{{?}}>11-32 (0.4 series)</option>
<option value="[37,99]" {{? master.proto_range=='[37,99]'}}selected{{?}}>37+ (5.0 or newer)</option>
</select>{{?}}
<div class="total">
Players: {{=it.total.clients}}/{{=it.total_max.clients}}&nbsp;
Servers: {{=it.total.servers}}/{{=it.total_max.servers}}
</div>
{{?}}
<table>
<thead><tr>
{{? !master.no_address}}<th>Address[:Port]</th>{{?}}
{{? !master.no_clients}}<th>Players / Max{{? !master.no_avgtop}}<br/>Average / Top{{?}}</th>{{?}}
{{? !master.no_version}}<th class="version">Version, Game, Mapgen</th>{{?}}
{{? !master.no_version}}<th>Version, Subgame, Mapgenerator</th>{{?}}
{{? !master.no_name}}<th>Name</th>{{?}}
{{? !master.no_description}}<th>Description</th>{{?}}
{{? !master.no_flags}}<th class="flags">Flags</th>{{?}}
{{? !master.no_uptime}}<th class="uptime">Uptime, Age</th>{{?}}
{{? !master.no_flags}}<th>Flags</th>{{?}}
{{? !master.no_uptime}}<th>Uptime, Age</th>{{?}}
{{? !master.no_ping}}<th>Ping, Lag</th>{{?}}
</tr></thead>
<tbody>
@@ -29,53 +21,56 @@
{{ if (master.min_clients && server.clients < master.min_clients) continue;}}
<tr>
{{? !master.no_address}}
<td class="address">
<td class ="address">
{{=addressString(server)}}
</td>{{?}}
{{? !master.no_clients}}
<td class="clients{{? server.clients_list && server.clients_list.length > 0}} mts_hover_list_text{{?}}">
{{=constantWidth(server.clients + '/' + server.clients_max, 3.4)}}
{{? !master.no_avgtop}} {{=constantWidth(Math.floor(server.pop_v) + '/' + server.clients_top, 3.4)}}{{?}}
{{=server.clients}}/{{=server.clients_max}}{{? !master.no_avgtop}} &nbsp;&nbsp;{{=Math.floor(server.pop_v)}}/{{=server.clients_top}}{{?}}
{{=hoverList("Clients", server.clients_list)}}
</td>{{?}}
{{? !master.no_version}}
<td class="version{{? server.mods && server.mods.length > 0}} mts_hover_list_text{{?}}">
{{!server.version}}, {{!server.gameid}}
{{? server.mapgen}}, {{!server.mapgen}}{{?}}
{{=escapeHTML(server.version)}}, {{=escapeHTML(server.gameid)}},&nbsp;
{{=escapeHTML(server.mapgen || '?')}}
{{=hoverList("Mods", server.mods)}}
</td>{{?}}
{{? !master.no_name}}
<td class="name">
{{? server.url}}
<a href="{{!server.url}}" target="_blank">{{=tooltipString(server.name)}}</a>
<a href="{{=escapeHTML(server.url)}}">{{=tooltipString(server.name, 25)}}</a>
{{??}}
{{=tooltipString(server.name)}}
{{=tooltipString(server.name, 25)}}
{{?}}
</td>{{?}}
{{? !master.no_description}}
<td class="description">
{{=tooltipString(server.description)}}
{{=tooltipString(server.description, 50)}}
</td>{{?}}
{{? !master.no_flags}}
<td class="flags {{? server.privs}} mts_hover_list_text{{?}}">
{{=hoverString("Default privileges", server.privs)}}
{{=hoverString("Privs", server.privs)}}
{{=server.creative ? 'Cre ' : ''}}
{{=server.dedicated ? 'Ded ' : ''}}
{{=server.damage ? 'Dmg ' : ''}}
{{=server.liquid_finite ? 'Liq ' : ''}}
{{=server.pvp ? 'PvP ' : ''}}
{{=server.password ? 'Pwd ' : ''}}
{{=server.rollback ? 'Rol ' : ''}}
{{=server.can_see_far_names ? 'Far ' : ''}}
</td>{{?}}
{{? !master.no_uptime}}
<td class="uptime">
{{=constantWidth(humanTime(server.uptime), 3.2)}} / {{=constantWidth(humanTime(server.game_time), 3.2)}}
{{=humanTime(server.uptime)}}, {{=humanTime(server.game_time)}}
</td>{{?}}
{{? !master.no_ping}}
<td class="ping">
{{=constantWidth(Math.floor(server.ping * 1000), 1.8)}}{{? server.lag}} / {{=constantWidth(Math.floor(server.lag * 1000), 1.8)}}{{?}}
{{=Math.floor(server.ping * 1000)}}{{? server.lag}}, {{= Math.floor(server.lag * 1000)}}{{?}}
</td>{{?}}
</tr>
{{~}}
</tbody>
</table>
{{? master.min_clients || master.limit}}
<a href="javascript:master.showAll()">Show all...</a>
<a class="clickable" onclick="delete master.min_clients; delete master.limit; get();">More...</a>
{{?}}

View File

@@ -1,4 +1,4 @@
#server_list .header_total {
#server_list .total {
font-weight: bold;
}
@@ -12,66 +12,42 @@
}
#server_list td, #server_list th {
border: 1px solid #2A3132;
padding: 5px;
color: inherit;
border: 1px solid gray;
}
#server_list thead {
background-color: #2A3132;
border-bottom: 5px solid #336B87;
color: white;
font-size: 1.1em;
background-color: #FFA;
}
#server_list tbody tr:nth-child(even) {
background-color: #E0E0E0;
background-color: #EEE;
}
#server_list td.clients, #server_list td.uptime, #server_list td.ping {
text-align: center;
}
#server_list td.version, #server_list td.name, #server_list td.description {
text-overflow: ellipsis;
overflow: hidden;
white-space: nowrap;
}
/* Note: the column widths here are not exact as auto-layout is left enabled */
#server_list td.address {
max-width: 24ch;
}
#server_list td.version {
max-width: 16ch;
}
#server_list td.name {
max-width: 32ch;
}
#server_list td.description {
max-width: 70ch;
#server_list tbody tr:hover {
background-color: #CCC;
}
.mts_hover_list {
display: none;
border: 1px solid #336B87;
border-radius: 10px;
background-color: #FFF;
border: 1px solid #88F;
border-radius: 4px;
background-color: white;
position: absolute;
z-index: 100;
padding: 0.5em;
box-shadow: 1px 1px 5px 3px rgba(0, 0, 0, 0.25);
}
.mts_hover_list b {
font-size: 1.1em;
}
td:hover .mts_hover_list {
display: block;
}
.mts_cwidth {
display: inline-block;
.mts_hover_list_text, .mts_tooltip {
text-decoration: underline;
text-decoration-style: dashed;
}
.clickable {
text-decoration: underline;
cursor: pointer;
}