Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support T-SQL OPENQUERY #43

Open
wants to merge 98 commits into
base: linked-servers-sp_dropserver-pg15
Choose a base branch
from

Conversation

thephantomthief
Copy link

@thephantomthief thephantomthief commented Jan 10, 2023

This commit adds support for T-SQL OPENQUERY rowset function. The function
has been implemented a table valued function. One limitation with table
valued functions is that we need to specify the exact schema of the expected
result set. Since OPENQUERY is a rowset function, the result set schema is
not fixed and can change depending on the parameters. To get around this
issue we have added a hook in the engine where we compute the tuple
descriptor from the SQL definition. If the function is OPENQUERY, we will
connect to the remote server and fetch the column metadata using
sp_describe_first_result_set stored procedure. Using this we modify the tuple
descriptor in the analyzer and execution phase so that it matches with the
actual result set when we run the actual query.

To communicate with remote servers using TDS, we are using the FreeTDS client
library APIs. In case someone does not want to compile source code with these
APIs, we have introduced a compile time flag "ENABLE_TDS_LIB". Calls to
FreeTDS APIs are surrounded with preprocessor directives that check for this
flag.

The testcases have been such that they reuse the existing tests we have T-SQL
data types. First, we run all the prepare scripts for the data types tests.
Next, instead of directly calling the SELECTs, they are all wrapped info
calls to OPENQUERY. Finally, we run all the cleanup scripts.

Sharu Goel and others added 30 commits December 19, 2022 14:19
This commit contains the following changes:
- Support for SELECTs via T-SQL OPENQUERY

Signed-off-by: Sharu Goel <[email protected]>
This information is needed for the SSMS scripting so that users have a way to view the
trigger definition in TSQL syntax and to complete the trigger definitions for the sql_modules view.

Task: BABEL-3681
Signed-off-by: Shalini Lohia <[email protected]>
…-postgresql#1060)

When a DDL is exported where an object's CHECK() clause has an explicit COLLATE clause with SQL Server collation, the collation name is converted to its equivalent BBF name while recreating the definition which is valid for BBF but not for SQL Server.

When a SQL Server collation is specified with an object in the CHECK() clause of a table, the definition will show the SQL Server collation name (there might be a possibility it shows it's closest match) but not it's equivalent Babelfish Collation name which would work on both SQL Server and BBF.
When a BBF collation name is provided, it shows it's closed SQL Server collation name if any. Otherwise it shows the same BBF name.

Task: BABEL-3838
Authored-by: Shalini Lohia <[email protected]>
Signed-off-by: Dipesh Dhameliya <[email protected]>
Following fixes have been done:
1. Fixed bounds of `strncpy` and `strncat` functions to not depend
upon source string, it is fix `stringop-overflow` compiler errors. ([Ref](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88059))
3. Replaced `strncpy` with `memcpy` at some places where use of
`strncpy` was not needed.
4. Added `pragma` to ignore some intentional GCC errors.

Signed-off-by: Rishabh Tanwar <[email protected]>
…ction name is used (babelfish-for-postgresql#1080)

While executing something like "SqlCommand("dbo.aspnet_CheckSchemaVersion", connection);",
pltsql_read_procedure_info() is called internally and it ignores any
schema mapping before calling FuncnameGetCandidates(). The code change
ensures to rewrite_object_refs() when necessary.

Task: BABEL-3530
Signed-off-by: Kristian Lejao <[email protected]>
The TIMESTAMP/ROWVERSION feature is off by default, and the corresponding escape hatch must be set to 'ignore' in order to switch on the feature.
However, the error messages that are raised need to be updated to be more correct and clear. Especially the mention of 'currently' is raising incorrect impressions.

Task: BABEL-3896
Signed-off-by: Shalini Lohia <[email protected]>
xhfanhe and others added 30 commits January 27, 2023 14:25
This commit re-designs UPDATE/DELETE DML statement support, including
but not restricted to
1. Plain UPDATE/DELETE
2. UPDATE/DELETE with JOIN (ANSI or non-ANSI, all types of JOIN)
3. UPDATE/DELETE with correlation name

Corresponding test cases are also added in this commit.

OUTPUT clause and TOP clause in UPDATE/DELETE will be supported in
future commits.

Task: BABEL-1330/1875/2675/3091/3684/3685/3775

Signed-off-by: Xiaohui Fanhe <[email protected]>
* Support STR() Transact-SQL function.

Task: BABAL-398
Signed-off-by: Yuhao Wei <[email protected]>

* Add str test to JDBC test schedule

Task: BABEL-398
Signed-off-by: Yuhao Wei <[email protected]>

* empty commit

Signed-off-by: Yuhao Wei <[email protected]>

* removed covered functions from expected_dependency.out

Signed-off-by: Yuhao Wei <[email protected]>

* removed some unused import from string.c

Task: BABEL-398
Signed-off-by: Yuhao Wei <[email protected]>

* Modify bracket convension

Task: BABEL-398
Signed-off-by: Yuhao Wei <[email protected]>

* Add test case for NaN and Infinity input

Task: BABEL-398
Signed-off-by: Yuhao Wei <[email protected]>

* empty commit

Signed-off-by: Yuhao Wei <[email protected]>

* Add some examples and explanations in the comments

Task: BABEL-398
Signed-off-by: Yuhao Wei <[email protected]>

* empty commit

Task: BABEL-398
Signed-off-by: Yuhao Wei <[email protected]>

* Seperate some parts of STR() function into several static helper
functions, add explanation and some test case about rounding in
integer part.

Task: BABEL-398
Signed-off-by: Yuhao Wei <[email protected]>

* add test str to 14_7/schedule

Task: BABEL-398
Signed-off-by: Yuhao Wei <[email protected]>

* empty commit

Signed-off-by: Yuhao Wei <[email protected]>

* empty commit

Signed-off-by: Yuhao Wei <[email protected]>

---------

Signed-off-by: Yuhao Wei <[email protected]>
Co-authored-by: Yuhao Wei <[email protected]>
…ostgresql#1017)

* Fix SET Statements to revert in Dynamic SQL

Currently, if a SET statement runs in a stored procedure or trigger, the
value of the option is restored to the previous value after the stored
procedure or trigger finishes executing. The same should apply for
dynamic SQL with both sp_executesql or EXEC, but it currently does not.
This change causes SET statements made in dynamic SQL to revert back to
their previous value once the dynamic statement has finished executing.

For example, sp_executesql N'SET DATEFIRST 1;' will cause datefirst to
be '1' inside of the executesql block, but will revert back to the
previous value once sp_executesql has finished.

The current change does not apply to all possible options.
IDENTITY_INSERT, PARSEONLY, SHOWPLAN_ALL, STATISTICS PROFILE,
TRANSACTION ISOLATION LEVEL, and BABELFISH_SHOWPLAN_ALL do not revert
correctly for both dynamic SQL and stored procedures.

Signed-off-by: Walt Boettge <[email protected]>

* Fix how GUCs are restored

Ensures that if we create a new babelfish guc level in a dynamic sql
block, we will revert that level even if we encountered an error.

Fixes an issue where new nest levels were being created without
restoring them.

Signed-off-by: Walt Boettge <[email protected]>

* Save global vars corresponding to SET options

Signed-off-by: Walt Boettge <[email protected]>

* Use postgres GUC stack to avoid improperly freeing GUC string settings

Signed-off-by: Walt Boettge <[email protected]>

* Remove changes to save / restore explain settings

Previously, explain family settings (along with parseonly) were manually
restored by saving their old values to the call stack. This
implementation isn't ideal, and they are not restored by stored
procedures either.

Instead, these settings will be fixed (for stored procedures as well) in
a future commit.

Affected settings are: showplan_all, babelfish_showplan_all, statistics
profile, babelfish_statistics profile, and parseonly.
These settings are not currently implemented as GUCs as they are set in
the parser, which is why they behave differently from other settings.

Signed-off-by: Walt Boettge <[email protected]>

* update tests

Signed-off-by: Walt Boettge <[email protected]>

* Fix final test case, missing 'go' statement

Signed-off-by: Walt Boettge <[email protected]>

* Revert back to using session_stack for GUCs

To fix an issue related to IDENTITY_INSERT, babelfish GUC revert logic
was changed to use the stack field, instead of the babel-specific
session stack. This caused a separate issue when reverting stored
procedures in different schemas. As a result, session_stack will once
again be used, and the identity_insert issue will be fixed by a separate
engine change.

Signed-off-by: Walt Boettge <[email protected]>

* Change engine branch to get tests passing

Signed-off-by: Walt Boettge <[email protected]>

* Re-trigger tests w/ empty commit

Signed-off-by: Walt Boettge <[email protected]>

* Empty commit

* Revert "Change engine branch to get tests passing"

This reverts commit 7514e53.

* Empty commit

Signed-off-by: Walt Boettge <[email protected]>

* Empty commit

Signed-off-by: Walt Boettge <[email protected]>

* Empty commit

Signed-off-by: Walt Boettge <[email protected]>

* Empty commit

Signed-off-by: Walt Boettge <[email protected]>

---------

Signed-off-by: Walt Boettge <[email protected]>
…#1146)

* BABEL-3914 Numeric Degrees/Radians Function (babelfish-for-postgresql#1140)

* BABEL-3914 Numeric Degrees/Radians Function

This Jira is a part of BABEL-736.
At first in Babel, the Transact-SQL functions degrees/radians always return different output instead of returning the same type in SQL server.
In this Jira we have added numeric to degrees/radians function.
Have added the test case to show numeric in degrees/radians function.
Now this is giving same output as SQL server.

Task: BABEL-3914
Signed-off-by: pratikzode <[email protected]>

* BABEL-3914 Resolved merge conflict

Signed-off-by: pratikzode <[email protected]>

* BABEL-3914 Resolved merge conflicts

Signed-off-by: pratikzode <[email protected]>

---------

Signed-off-by: pratikzode <[email protected]>
…ql#1133)

Description
Due to unstable/flaky behaviour apt-spy2 and longer install dependency times ,
The new approach downloads a package from 10-20 mirrors , sort them by download speed and picks the fastest mirror for download , also has logic for retries set 10 .

example run :- link

Signed-off-by: Nirmit Shah [email protected]
In SQL Server, User can provide partial domain name/ Netbios domain name in all DDL related to WINDOWS LOGIN (CREATE/ALTER/DROP LOGIN). SQL Server can still figure out the full domain name from the AD. For example, BABEL\tom, BABEL.COM\tom, adtest.BABEL.COM\tom all are valid in SQL Server.

It is not possible to translate this netbios domain name to fully qualified domain name without knowing the LDAP credential of AD setup. So, In Babelfish (and in PG), the engine doesn't have the ability to automatically figure out the full domain name from partial domain name.

To resolve, We need create domain mapping catalog which can maintain this Netbios domain name to FQDN. This commit introduces new catalog sys.babelfish_domain_mapping to hold the Netbios domain name to FQDN mapping. As well as introduces procedures, sys.babelfish_add_domain_mapping_entry to add domain mapping entry, sys.babelfish_remove_domain_mapping_entry to remove domain mapping entry and  sys.babelfish_reset_domain_mapping to reset the domain mapping entries.

With this change, when convertToUPN() is called to convert the user format, convertToUPN will internally call get_fully_qualified_domain_name() to retrieve FQDN from customer provided Netbios domain name.

For example, If user has added [babel] to [babel.internal] mapping and If customer uses [babel\test] login then it will internally converted to [email protected]. 

Task: BABEL-3863, BABEL-3847
Signed-off-by: Dipesh Dhameliya <[email protected]>
…resql#1088)

This commit adds support for sp_linkedservers stored procedure. sp_linkedservers has been implemented such that internally it uses sys.servers view.

Signed-off-by: Sai Rohan Basa <[email protected]>
This commit disallows '\\' in password based login, role name and user name. '\\' is only allowed for windows based login. This commit also updates DROP LOGIN logic to use convertToUPN() and removes the use of get_roleform_ext() as we can now safely assume that given login is windows based login if it contains '\\'. 

Task: BABEL-3828, BABEL-3844, BABEL-3847
Signed-off-by: Shameem Ahmed <[email protected]>
…-postgresql#1102)

Previously The underlying view used in sp_fkeys procedure was written in such a way that it had multiple joins between logical views (information schema views) which led to optimizer rewriting the view query into rather a slow query which lead to sequential scans( also led to high cpu usage ) over whole pg_attribute table which have entries/rows proportional to objects(tables) present in the database , this lead to degradation of runtime of query with large database which is evident from the github action runs timing out in past.

The new rewrite of view aims to use the pg_catalog tables instead of logical views where ever possible and joins are done in such a way that index scan is given preference over sequential scan . The new view also adds Deferrability column to the view which is expected in result set for sp_fkeys.

Authored-by: Nirmit Shah <[email protected]>
Signed-off-by: Kushaal Shroff <[email protected]>
…sql#1117)

This commit adds support of sys.systypes system compatibility view which returns one row for each system-supplied and each user-defined data type defined in the database.

Note: xtype/type columns are internally pg_type.oid (4 byte int). Casting them to tinyint might cause overflow.

Signed-off-by: Sumit Jaiswal [email protected]
…stgresql#1161)

This commit supports ALTER LOGIN for windows based login.
Here, we are leveraging the fact that convertToUPN API returns the login name in UPN format if login name contains '\' i,e,. windows login. For windows login '\' must be present and for password based login '\' is not acceptable. So, combining these, if the login is from windows then it will be converted to UPN format or else it will be as it was.

Task: BABEL-3847, BABEL-3882
Signed-off-by: Shameem Ahmed <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.