Skip to content

TestPreprocessor: removed usage of std::istringstream #6873

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 22, 2025
Merged

TestPreprocessor: removed usage of std::istringstream #6873

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 24 additions & 26 deletions test/testpreprocessor.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@
#include <list>
#include <map>
#include <set>
#include <sstream>
#include <stdexcept>
#include <string>
#include <utility>
Expand All @@ -50,27 +49,27 @@ class TestPreprocessor : public TestFixture {
TestPreprocessor() : TestFixture("TestPreprocessor") {}

private:
std::string expandMacros(const char code[], ErrorLogger &errorLogger) const {
std::istringstream istr(code);
template<size_t size>
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the advantage at all here neither. What exactly is the advantage?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not need the stream and get the size.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not very convinced about this. It feels like using a template and complicating the code just for fun.
imho it's a bit weird syntax const char (&code)[size] we could have used a strlen or a std::string. this is test code.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not for fun. Some code in the core is just provided to accommodate the tests and that should be get rid of (and will be done in other changes).

And we run the test code mostly in debug i.e. no optimizations so it is slower than production code and thus having slower code for convenience should be avoided. Makes me wonder if unoptimized matchcompiled code might behave better.

std::string expandMacros(const char (&code)[size], ErrorLogger &errorLogger) const {
simplecpp::OutputList outputList;
std::vector<std::string> files;
const simplecpp::TokenList tokens1 = simplecpp::TokenList(istr, files, "file.cpp", &outputList);
const simplecpp::TokenList tokens1 = simplecpp::TokenList(code, size-1, files, "file.cpp", &outputList);
Preprocessor p(settingsDefault, errorLogger, Path::identify(tokens1.getFiles()[0], false));
simplecpp::TokenList tokens2 = p.preprocess(tokens1, "", files, true);
p.reportOutput(outputList, true);
return tokens2.stringify();
}

static void preprocess(const char code[], std::vector<std::string> &files, const std::string& file0, TokenList& tokenlist, const simplecpp::DUI& dui)
template<size_t size>
static void preprocess(const char (&code)[size], std::vector<std::string> &files, const std::string& file0, TokenList& tokenlist, const simplecpp::DUI& dui)
{
if (!files.empty())
throw std::runtime_error("file list not empty");

if (tokenlist.front())
throw std::runtime_error("token list not empty");

std::istringstream istr(code);
const simplecpp::TokenList tokens1(istr, files, file0);
const simplecpp::TokenList tokens1(code, size-1, files, file0);

// Preprocess..
simplecpp::TokenList tokens2(files);
Expand All @@ -82,11 +81,11 @@ class TestPreprocessor : public TestFixture {
tokenlist.createTokens(std::move(tokens2));
}

std::vector<RemarkComment> getRemarkComments(const char code[], ErrorLogger& errorLogger) const
template<size_t size>
std::vector<RemarkComment> getRemarkComments(const char (&code)[size], ErrorLogger& errorLogger) const
{
std::vector<std::string> files;
std::istringstream istr(code);
const simplecpp::TokenList tokens1(istr, files, "test.cpp");
const simplecpp::TokenList tokens1(code, size-1, files, "test.cpp");

const Preprocessor preprocessor(settingsDefault, errorLogger, Path::identify(tokens1.getFiles()[0], false));
return preprocessor.getRemarkComments(tokens1);
Expand Down Expand Up @@ -301,16 +300,16 @@ class TestPreprocessor : public TestFixture {
TEST_CASE(standard);
}

std::string getConfigsStr(const char filedata[], const char *arg = nullptr) {
template<size_t size>
std::string getConfigsStr(const char (&code)[size], const char *arg = nullptr) {
Settings settings;
if (arg && std::strncmp(arg,"-D",2)==0)
settings.userDefines = arg + 2;
if (arg && std::strncmp(arg,"-U",2)==0)
settings.userUndefs.insert(arg+2);
std::vector<std::string> files;
std::istringstream istr(filedata);
// TODO: this adds an empty filename
simplecpp::TokenList tokens(istr,files);
simplecpp::TokenList tokens(code, size-1,files);
tokens.removeComments();
Preprocessor preprocessor(settings, *this, Standards::Language::C); // TODO: do we need to consider #file?
const std::set<std::string> configs = preprocessor.getConfigs(tokens);
Expand All @@ -320,11 +319,11 @@ class TestPreprocessor : public TestFixture {
return ret;
}

std::size_t getHash(const char filedata[]) {
template<size_t size>
std::size_t getHash(const char (&code)[size]) {
std::vector<std::string> files;
std::istringstream istr(filedata);
// TODO: this adds an empty filename
simplecpp::TokenList tokens(istr,files);
simplecpp::TokenList tokens(code,size-1,files);
tokens.removeComments();
Preprocessor preprocessor(settingsDefault, *this, Standards::Language::C); // TODO: do we need to consider #file?
return preprocessor.calculateHash(tokens, "");
Expand Down Expand Up @@ -478,9 +477,8 @@ class TestPreprocessor : public TestFixture {
"#else\n"
"2\n"
"#endif\n";
std::istringstream istr(filedata);
std::vector<std::string> files;
simplecpp::TokenList tokens(istr, files, "test.c");
simplecpp::TokenList tokens(filedata, sizeof(filedata), files, "test.c");

// preprocess code with unix32 platform..
{
Expand Down Expand Up @@ -803,14 +801,14 @@ class TestPreprocessor : public TestFixture {
}

void if_macro_eq_macro() {
const char *code = "#define A B\n"
"#define B 1\n"
"#define C 1\n"
"#if A == C\n"
"Wilma\n"
"#else\n"
"Betty\n"
"#endif\n";
const char code[] = "#define A B\n"
"#define B 1\n"
"#define C 1\n"
"#if A == C\n"
"Wilma\n"
"#else\n"
"Betty\n"
"#endif\n";
ASSERT_EQUALS("\n", getConfigsStr(code));
}

Expand Down
Loading