tonytins/cstdotnet
1
0
Fork 0
mirror of https://github.com/tonytins/cstdotnet.git synced 2025-03-15 14:11:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

1.0.2 (#2)

Browse source 
- Fixed multiple line parsing errors in the v2 format.
- Replaced "[ENTRY NOT FOUND]" message with "***MISSING***".
This commit is contained in:
Tony Bark 2020-12-12 04:03:00 -05:00 • committed by GitHub
commit e8f7bf4d0a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
9 changed files with 64 additions and 108 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
CSTNet.Tests/QuickFennec.CST.Tests.csproj → CSTNet.Tests/Sixam.CST.Tests.csproj
View file

@ -19,7 +19,7 @@
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\CSTNet\QuickFennec.CST.csproj" />
<ProjectReference Include="..\CSTNet\Sixam.CST.csproj" />
</ItemGroup>
<ItemGroup>

2
CSTNet.Tests/v2.cst
View file

@ -1,4 +1,4 @@
Singleline^Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed ultricies nulla eu tortor mattis, dictum posuere lacus ornare. Maecenas a massa in ligula finibus luctus eu vitae nibh. Proin imperdiet dapibus mauris quis placerat.^
Multiline ^Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc gravida nunc non justo pretium consectetur. Sed tempus libero ac ligula aliquam elementum. Duis vitae interdum leo. Sed semper nulla %1 a lectus dictum dictum.
Quisque vehicula, nisi ut scelerisque sodales, nisi ipsum sodales ipsum, in rutrum tellus lacus sed nibh. Etiam mauris velit, elementum sed placerat et, elementum et tellus. Duis vitae elit fermentum, viverra lorem in, lobortis elit.^
Quisque vehicula, nisi ut scelerisque sodales, nisi ipsum sodales ipsum, in rutrum tellus lacus sed nibh. Etiam mauris velit, elementum sed placerat et, elementum et tellus. Duis vitae elit fermentum, viverra lorem in, lobortis elit^

62
CSTNet/CaretSeparatedText.cs
View file

@ -38,46 +38,36 @@ namespace CSTNet
/// <remarks>This stage ensures there are no crashes during parsing.</remarks>
static IEnumerable<string> NormalizeEntries(string content)
{
/*
I tried putting the end carets with the different
line endings in with the split function but it didn't work
*/
if (!content.Contains($"{CARET}{Environment.NewLine}"))
if (!content.Contains(Environment.NewLine))
{
if (content.Contains($"{CARET}{_lf}"))
content = content.Replace($"{CARET}{_lf}",
$"{CARET}{Environment.NewLine}");
if (content.Contains(_lf))
content = content.Replace(_lf, Environment.NewLine);
if (content.Contains($"{CARET}{_cr}"))
content = content.Replace($"{CARET}{_cr}",
$"{CARET}{Environment.NewLine}");
if (content.Contains(_cr))
content = content.Replace(_cr, Environment.NewLine);
if (content.Contains($"{CARET}{_crlf}"))
content = content.Replace($"{CARET}{_crlf}",
$"{CARET}{Environment.NewLine}");
if (content.Contains(_crlf))
content = content.Replace(_crlf, Environment.NewLine);
if (content.Contains($"{CARET}{_ls}"))
content = content.Replace($"{CARET}{_ls}",
$"{CARET}{Environment.NewLine}");
if (content.Contains(_ls))
content = content.Replace(_ls, Environment.NewLine);
}
var entries = content.Split(new[] { $"{CARET}{Environment.NewLine}" },
var lines = content.Split(new[] { $"{CARET}{Environment.NewLine}" },
StringSplitOptions.RemoveEmptyEntries);
var newContent = new List<string>();
var entries = new List<string>();
foreach (var entry in entries)
foreach (var line in lines)
{
// Skip comments
if (entry.StartsWith(@"//") || entry.StartsWith("#") ||
entry.StartsWith("/*") || entry.EndsWith("*/"))
if (line.StartsWith("//") || line.StartsWith("#") ||
line.StartsWith("/*") || line.EndsWith("*/"))
continue;
newContent.Add(entry);
entries.Add(line);
}
return newContent;
return entries;
}
static string GetEntry(IEnumerable<string> entries, string key)
@ -85,32 +75,18 @@ namespace CSTNet
// Search through list
foreach (var entry in entries)
{
// Locate index, trim carets and return translation
// If the line doesn't start with the key, keep searching.
if (!entry.StartsWith(key))
continue;
// Locate index, trim carets and return translation.
var startIndex = entry.IndexOf(CARET);
var line = entry.Substring(startIndex);
if (!line.Contains(Environment.NewLine))
{
if (line.Contains(_lf))
line = line.Replace(_lf, Environment.NewLine);
if (line.Contains(_cr))
line = line.Replace(_cr, Environment.NewLine);
if (line.Contains(_crlf))
line = line.Replace(_crlf, Environment.NewLine);
if (line.Contains(_ls))
line = line.Replace(_ls, Environment.NewLine);
}
return line.TrimStart(CARET).TrimEnd(CARET);
}
return "[ENTRY NOT FOUND]";
return "***MISSING***";
}
}
}

4
CSTNet/QuickFennec.CST.csproj → CSTNet/Sixam.CST.csproj
View file

@ -2,10 +2,10 @@
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
<Version>1.0.1</Version>
<Version>1.0.2</Version>
<Authors>Tony Bark</Authors>
<PackageDescription>Caret-Separated Text (or CST) is a key-value pair format represented by numbers or words as keys and the value is the string enclosed between carets (^) that contains the contents. CST.NET is a library for prasing the CST format.</PackageDescription>
<RepositoryUrl>https://github.com/tonytins/cstnet</RepositoryUrl>
<RepositoryUrl>https://github.com/sixamsoft/cst-dotnet</RepositoryUrl>
<PackageLicenseExpression>MIT</PackageLicenseExpression>
<Product>CST.Net</Product>
<AssemblyName>CSTNet</AssemblyName>

23
README.md
View file

@ -1,15 +1,17 @@
# QuickFennec.CST
# Sixam.CST
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg)](code_of_conduct.md)
Caret-Separated Text (or CST) is a key-value pair format represented by numbers or words as keys and the value is the string enclosed between carets (^) that contains the contents. Any text which is not enclosed with carets is considered a comment and ignored. Neither strings nor comments may use the caret character.
QuickFennec.CST is a library for parsing the CST format.
Caret-Separated Text (or CST) is a key-value pair format represented by digits or words as keys and the value as text enclosed between carets. (e.g. ``<key> ^<text>^``) Any text which is not enclosed with carets is considered a comment and ignored. Neither strings nor comments may use the caret character. Sixam.CST is a library for parsing the CST format.
## Usage
```text
1 ^The quick brown fox jumps over the lazy dog.^
```
```csharp
#r "nuget:CSTNet,1.0.1"
#r "nuget:CSTNet,1.0.2"
using System;
using System.IO;
using CSTNet;
@ -20,12 +22,14 @@ var example = CaretSeparatedText.Parse(file, 1);
Console.WriteLine(example);
```
In production, CST files were used in The Sims Online to provide translations. Each translation was split into their respective directories:
See working example on [.NET Fiddle](https://dotnetfiddle.net/ecKb2h).
- ``uitext/english.dir/misc/_154_miscstrings.cst``
- ``uitext/swedish.dir/misc/_154_miscstrings.cst``
In production, CST files were used in The Sims Online (TSO) to provide translations. Each translation was split into their respective directories:
QuickFennec.CST only provides the basic parsing functionality.
- ``uitext/english.dir/_154_miscstrings.cst``
- ``uitext/swedish.dir/_154_miscstrings.cst``
Sixam.CST only provides the basic parsing functionality.
## To-do
@ -34,7 +38,6 @@ QuickFennec.CST only provides the basic parsing functionality.
## Known issues
- Skipping comments is a little buggy.
- Multiline parsing with the v2 format is still unpredictable.
## Requirements
### Prerequisites

4
CSTNet.sln → Sixam.CST.sln
View file

@ -3,7 +3,7 @@ Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 16
VisualStudioVersion = 16.0.30717.126
MinimumVisualStudioVersion = 10.0.40219.1
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "QuickFennec.CST", "CSTNet\QuickFennec.CST.csproj", "{82775826-A366-46F0-A5D2-5BE7658C75E4}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Sixam.CST", "CSTNet\Sixam.CST.csproj", "{82775826-A366-46F0-A5D2-5BE7658C75E4}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{CCFCE2DB-C18F-4D88-B025-19ED62BD2A1D}"
ProjectSection(SolutionItems) = preProject
@ -12,7 +12,7 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution
README.md = README.md
EndProjectSection
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "QuickFennec.CST.Tests", "CSTNet.Tests\QuickFennec.CST.Tests.csproj", "{B6A98C64-1419-4B9A-99CA-72BB11D29472}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Sixam.CST.Tests", "CSTNet.Tests\Sixam.CST.Tests.csproj", "{B6A98C64-1419-4B9A-99CA-72BB11D29472}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution

5
changelog.md
View file

@ -1,5 +1,8 @@
# Change Log
## 1.0.2
- Fixed the multiple line parsing in the v2 format.
- Replaced "``[ENTRY NOT FOUND]``" message with "``***MISSING***``".
## 1.0.1
Despite only being a point release, this includes a major refinement to the normalizing algorithm.
@ -10,7 +13,7 @@ The normalizing algorithm has been rewritten to be more efficient and hopefully
### CSTNet compatibility
For point releases (such as this), QuickFennec.CST will remain under the CSTNet namespace for compatibility reasons. CSTNet will be moved to QuickFennec.CST namespace starting with 1.1.
For point releases (such as this), Sixam.CST will remain under the CSTNet namespace for compatibility reasons. CSTNet will be moved to Sixam.CST namespace starting with 1.1.
### Known issues

64
notebooks/cst.ipynb
View file

@ -63,46 +63,36 @@
" /// <remarks>This stage ensures there are no crashes during parsing.</remarks>\n",
" static IEnumerable<string> NormalizeEntries(string content)\n",
" {\n",
"\n",
" /* \n",
" I tried putting the end carets with the different\n",
" line endings in with the split function but it didn't work \n",
" */\n",
" if (!content.Contains($\"{CARET}{Environment.NewLine}\"))\n",
" if (!content.Contains(Environment.NewLine))\n",
" {\n",
" if (content.Contains($\"{CARET}{_lf}\"))\n",
" content = content.Replace($\"{CARET}{_lf}\",\n",
" $\"{CARET}{Environment.NewLine}\");\n",
" if (content.Contains(_lf))\n",
" content = content.Replace(_lf, Environment.NewLine);\n",
"\n",
" if (content.Contains($\"{CARET}{_cr}\"))\n",
" content = content.Replace($\"{CARET}{_cr}\",\n",
" $\"{CARET}{Environment.NewLine}\");\n",
" if (content.Contains(_cr))\n",
" content = content.Replace(_cr, Environment.NewLine);\n",
"\n",
" if (content.Contains($\"{CARET}{_crlf}\"))\n",
" content = content.Replace($\"{CARET}{_crlf}\",\n",
" $\"{CARET}{Environment.NewLine}\");\n",
" if (content.Contains(_crlf))\n",
" content = content.Replace(_crlf, Environment.NewLine);\n",
"\n",
" if (content.Contains($\"{CARET}{_ls}\"))\n",
" content = content.Replace($\"{CARET}{_ls}\",\n",
" $\"{CARET}{Environment.NewLine}\");\n",
" if (content.Contains(_ls))\n",
" content = content.Replace(_ls, Environment.NewLine);\n",
" }\n",
"\n",
"\n",
" var entries = content.Split(new[] { $\"{CARET}{Environment.NewLine}\" },\n",
" var lines = content.Split(new[] { $\"{CARET}{Environment.NewLine}\" },\n",
" StringSplitOptions.RemoveEmptyEntries);\n",
" var newContent = new List<string>();\n",
" var entries = new List<string>();\n",
"\n",
" foreach (var entry in entries)\n",
" foreach (var line in lines)\n",
" {\n",
" // Skip comments\n",
" if (entry.StartsWith(@\"//\") || entry.StartsWith(\"#\") ||\n",
" entry.StartsWith(\"/*\") || entry.EndsWith(\"*/\"))\n",
" if (line.StartsWith(\"//\") || line.StartsWith(\"#\") ||\n",
" line.StartsWith(\"/*\") || line.EndsWith(\"*/\"))\n",
" continue;\n",
"\n",
" newContent.Add(entry);\n",
" entries.Add(line);\n",
" }\n",
"\n",
" return newContent;\n",
" return entries;\n",
" }\n",
"\n",
" static string GetEntry(IEnumerable<string> entries, string key)\n",
@ -110,32 +100,18 @@
" // Search through list\n",
" foreach (var entry in entries)\n",
" {\n",
" // Locate index, trim carets and return translation\n",
" // If the line doesn't start with the key, keep searching.\n",
" if (!entry.StartsWith(key))\n",
" continue;\n",
"\n",
" // Locate index, trim carets and return translation.\n",
" var startIndex = entry.IndexOf(CARET);\n",
" var line = entry.Substring(startIndex);\n",
"\n",
" if (!line.Contains(Environment.NewLine))\n",
" {\n",
" if (line.Contains(_lf))\n",
" line = line.Replace(_lf, Environment.NewLine);\n",
"\n",
" if (line.Contains(_cr))\n",
" line = line.Replace(_cr, Environment.NewLine);\n",
"\n",
" if (line.Contains(_crlf))\n",
" line = line.Replace(_crlf, Environment.NewLine);\n",
"\n",
" if (line.Contains(_ls))\n",
" line = line.Replace(_ls, Environment.NewLine);\n",
" }\n",
"\n",
" return line.TrimStart(CARET).TrimEnd(CARET);\n",
" }\n",
"\n",
" return \"[ENTRY NOT FOUND]\";\n",
" return \"***MISSING***\";\n",
" }\n",
"}"
],
@ -206,7 +182,7 @@
{
"output_type": "execute_result",
"data": {
"text/plain": "Multiline v2:\r\n[ENTRY NOT FOUND]\r\n"
"text/plain": "Multiline v2:\r\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc gravida nunc non justo pretium consectetur. Sed tempus libero ac ligula aliquam elementum. Duis vitae interdum leo. Sed semper nulla %1 a lectus dictum dictum.\r\n\r\nQuisque vehicula, nisi ut scelerisque sodales, nisi ipsum sodales ipsum, in rutrum tellus lacus sed nibh. Etiam mauris velit, elementum sed placerat et, elementum et tellus. Duis vitae elit fermentum, viverra lorem in, lobortis elit\r\n"
},
"execution_count": 1,
"metadata": {}

6
notebooks/data/v2.cst
View file

@ -1,6 +1,4 @@
Singleline^Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed ultricies nulla eu tortor mattis, dictum posuere lacus ornare. Maecenas a massa in ligula finibus luctus eu vitae nibh. Proin imperdiet dapibus mauris quis placerat.^
/* this is a
test comment */
Multiline^Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc gravida nunc non justo pretium consectetur. Sed tempus libero ac ligula aliquam elementum. Duis vitae interdum leo. Sed semper nulla %1 a lectus dictum dictum. Ut mattis eu tortor in bibendum. Integer mattis tincidunt aliquet. Vestibulum ante ipsum primis in faucibus orci %2 luctus et ultrices posuere cubilia Curae; Fusce quis orci nisl.
Multiline ^Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc gravida nunc non justo pretium consectetur. Sed tempus libero ac ligula aliquam elementum. Duis vitae interdum leo. Sed semper nulla %1 a lectus dictum dictum.
Quisque vehicula, nisi ut scelerisque sodales, nisi ipsum sodales ipsum, in rutrum tellus lacus sed nibh. Etiam mauris velit, elementum sed placerat et, elementum et tellus. Duis vitae elit fermentum, viverra lorem in, lobortis elit. Maecenas eget nibh et lectus auctor dignissim.^
Quisque vehicula, nisi ut scelerisque sodales, nisi ipsum sodales ipsum, in rutrum tellus lacus sed nibh. Etiam mauris velit, elementum sed placerat et, elementum et tellus. Duis vitae elit fermentum, viverra lorem in, lobortis elit^