Strings Mangled through Custom Function

[prologician]

I'm using ruby 2.1.7p400 (2015-08-18 revision 51632) [i386-mingw32], and have the gem sqlite3 (1.3.13 x86-mingw32) installed. As part of a larger Ruby script, I was doing some database manipulations. While my main development environment ran fine, an alternate environment was seeing issues when processing data fetched from one table and inserted into another. The following test exercises the issue:

#!/usr/bin/ruby
# encoding: UTF-8

require 'sqlite3'
require 'minitest/autorun'

class TestCreateFunction < MiniTest::Unit::TestCase
    def setup
        @db = SQLite3::Database.new('')
        @db.execute("CREATE TABLE
                       sourceTable(
                         sourceData TEXT);")
        @db.execute("INSERT INTO sourceTable
                     VALUES ('abcde');")

        @db.create_function("GetCopy", 1) {|func, value|
            func.result = value
        }
    end

    def test_passthrough_udf
        @db.transaction {|t|
            t.execute("CREATE TABLE
                         afterTable(
                           beforeData TEXT,
                           afterData TEXT);".squeeze(" "))

            t.execute("INSERT INTO afterTable
                       SELECT
                         sourceData,
                         GetCopy(sourceData)
                       FROM sourceTable;")
        }

        refute_nil(@db.get_first_value("SELECT 1
                                        FROM afterTable
                                        WHERE beforeData = afterData
                                        LIMIT 1;"))
    end
end

The above test fails when using the sqlite3 gem 1.3.13, but passes with version 1.3.11.

When using a nonempty database name and inspecting the values in afterTable, it appears that the output of GetCopy() generated a blob instead of preserving the properties of the original string.

[flavorjones]

Thank you for the executable test! I git-bisected and found that 44860d4 from #169 caused this test to start failing.

Reading the PR, this seems like a sensible change to make -- ASCII-8BIT encoding essentially means "binary encoding" and so a BLOB is arguably the correct data type for a string with this encoding.

However, given you're manipulating ordinary text I'm wondering why that encoding is being used, and perhaps that's the deeper issue we should look into.