Add support to readelf and objdump for following links to separate debug information files.

Hi Guys,

  I am applying the rather large patch attached to this email to enhance
  the readelf and objdump programs so that they now have the ability to
  follow links to separate debug info files.  (As requested by PR
  15152).  So for example whereas before we had this output:

    $ readelf -wi main.exe

    Contents of the .debug_info section:
    [...]
    <15>   DW_AT_comp_dir    : (alt indirect string, offset: 0x30c)
    [...]

  With the new option enabled we get:

    $ readelf -wiK main.exe

    main.exe: Found separate debug info file: dwz.debug
    Contents of the .debug_info section (loaded from main.exe):
    [...]
    <15>   DW_AT_comp_dir    : (alt indirect string, offset: 0x30c) /home/nickc/Downloads/dwzm
    [...]

  The link following feature also means that we can get two lots of
  output if the same section exists in both the main file and the
  separate debug info file:

    $ readelf -wiK main.exe
    main.exe: Found separate debug info file: dwz.debug
    Contents of the .debug_info section (loaded from main.exe):
    [...]
    Contents of the .debug_info section (loaded from dwz.debug):
    [...]

  The patch also adds the ability to display the contents of debuglink
  sections:

    $ readelf -wk main.exe
    Contents of the .gnu_debugaltlink section:

      Separate debug info file: dwz.debug
      Build-ID (0x14 bytes):
     c4 a8 89 8d 64 cf 70 8a 35 68 21 f2 ed 24 45 3e 18 7a 7a 93

  Naturally there are long versions of these options (=follow-links and
  =links).  The documentation has been updated as well, and since both
  readelf and objdump use the same set of debug display options, I have
  moved the text into a separate file.  There are also a couple of new
  binutils tests to exercise the new behaviour.

  There are a couple of missing features in the current patch however,
  although I do intend to address them in follow up submissions:

  Firstly the code does not check the build-id inside separate debug
  info files when it is searching for a file specified by a
  .gnu_debugaltlink section.  It just assumes that if the file is there,
  then it contains the information being sought.

  Secondly I have not checked the DWARF-5 version of these link
  features, so there will probably be code to add there.

  Thirdly I have only implemented link following for the
  DW_FORM_GNU_strp_alt format.  Other alternate formats (eg
  DW_FORM_GNU_ref_alt) have yet to be implemented.

  Lastly, whilst implementing this feature I found it necessary to move
  some of the global variables used by readelf (eg section_headers) into
  a structure that can be passed around.  I have moved all of the global
  variables that were necessary to get the patch working, but I need to
  complete the operation and move the remaining, file-specific variables
  (eg dynamic_strings).

Cheers
  Nick

binutils	PR 15152
	* dwarf.h (enum dwarf_section_display_enum): Add gnu_debuglink,
	gnu_debugaltlink and separate_debug_str.
	(struct dwarf_section): Add filename field.
	Add prototypes for load_separate_debug_file, close_debug_file and
	open_debug_file.
	* dwarf.c (do_debug_links): New.
	(do_follow_links): New.
	(separate_debug_file, separate_debug_filename): New.
	(fetch_alt_indirect_string): New function.  Retrieves a string
	from the debug string table in the separate debug info file.
	(read_and_display_attr_value): Use it with DW_FORM_GNU_strp_alt.
	(load_debug_section_with_follow): New function.  Like
	load_debug_section, but if the first attempt fails, then tries
	again in the separate debug info file.
	(introduce): New function.
	(process_debug_info): Use load_debug_section_with_follow and
	introduce.
	(load_debug_info): Likewise.
	(display_debug_lines_raw): Likewise.
	(display_debug_lines_decoded): Likewise.
	(display_debug_macinfo): Likewise.
	(display_debug_macro): Likewise.
	(display_debug_abbrev): Likewise.
	(display_debug_loc): Likewise.
	(display_debug_str): Likewise.
	(display_debug_aranges): Likewise.
	(display_debug_addr); Likewise.
	(display_debug_frames): Likewise.
	(display_gdb_index): Likewise.
	(process_cu_tu_index): Likewise.
	(load_cu_tu_indexes): Likewise.
	(display_debug_links): New function.  Displays the contents of a
	.gnu_debuglink or .gnu_debugaltlink section.
	(calc_gnu_debuglink_ctc32):New function.  Calculates a CRC32
	value.
	(check_gnu_debuglink): New function.  Checks the CRC of a
	potential separate debug info file.
	(parse_gnu_debuglink): New function.  Reads a CRC value out of a
	.gnu_debuglink section.
	(check_gnu_debugaltlink): New function.
	(parse_gnu_debugaltlink): New function.  Reads the build-id value
	out of a .gnu_debugaltlink section.
	(load_separate_debug_info): New function.  Finds and loads a
	separate debug info file.
	(load_separate_debug_file): New function. Attempts to find and
	follow a link to a separate debug info file.
	(free_debug_memory): Free the separate debug info file
	information.
	(opts_table): Add "follow-links" and "links".
	(dwarf_select_sections_by_letters): Add "k" and "K".
	(debug_displays): Reformat.  Add .gnu-debuglink and
	.gnu_debugaltlink.
	Add an extra entry for .debug_str in a separate debug info file.
	* doc/binutils.texi: Move description of debug dump features
	common to both readelf and objdump into...
	* objdump.c (usage): Add -Wk and -WK.
	(load_specific_debug_section): Initialise the filename field in
	the dwarf_section structure.
	(close_debug_file): New function.
	(open_debug_file): New function.
	(dump_dwarf): Load and dump the separate debug info sections.
	* readelf.c (struct filedata): New structure.  Contains various
	variables that used to be global:
	(current_file_size, string_table, string_table_length, elf_header)
	(section_headers, program_headers, dump_sects, num_dump_sects):
	Move into filedata structure.
	(cmdline): New global variable.  Contains list of sections to dump
	by number, as specified on the command line.
	Add filedata parameter to most functions.
	(load_debug_section): Load the string table if it has not already
	been retrieved.
	(close_file): New function.
	(close_debug_file): New function.
	(open_file): New function.
	(open_debug_file): New function.
	(process_object): Process sections in any separate debug info files.
	* doc/debug.options.texi: New file.  Add description of =links and
	=follow-links options.
	* NEWS: Mention the new feature.
	* elfcomm.c: Have the byte gte functions take a const pointer.
	* elfcomm.h: Update prototypes.
	* testsuite/binutils-all/dw5.W: Update expected output.
	* testsuite/binutils-all/objdump.WL: Update expected output.
	* testsuite/binutils-all/objdump.exp: Add test of -WK and -Wk.
	* testsuite/binutils-all/readelf.exp: Add test of -wK and -wk.
	* testsuite/binutils-all/readelf.k: New file.
	* testsuite/binutils-all/objdump.Wk: New file.
	* testsuite/binutils-all/objdump.WK2: New file.
	* testsuite/binutils-all/linkdebug.s: New file.
	* testsuite/binutils-all/debuglink.s: New file.

gas	* testsuite/gas/avr/large-debug-line-table.d: Update expected
	output.
	* testsuite/gas/elf/dwarf2-11.d: Likewise.
	* testsuite/gas/elf/dwarf2-12.d: Likewise.
	* testsuite/gas/elf/dwarf2-13.d: Likewise.
	* testsuite/gas/elf/dwarf2-14.d: Likewise.
	* testsuite/gas/elf/dwarf2-15.d: Likewise.
	* testsuite/gas/elf/dwarf2-16.d: Likewise.
	* testsuite/gas/elf/dwarf2-17.d: Likewise.
	* testsuite/gas/elf/dwarf2-18.d: Likewise.
	* testsuite/gas/elf/dwarf2-5.d: Likewise.
	* testsuite/gas/elf/dwarf2-6.d: Likewise.
	* testsuite/gas/elf/dwarf2-7.d: Likewise.

ld	* testsuite/ld-avr/gc-section-debugline.d: Update expected
	output.

9 files changed, 278 insertions, 2 deletions

diff --git a/binutils/testsuite/binutils-all/debuglink.s b/binutils/testsuite/binutils-all/debuglink.s
new file mode 100644
index 0000000000..a3691a03c3
--- /dev/null
+++ b/binutils/testsuite/binutils-all/debuglink.s
@@ -0,0 +1,67 @@
+/* Assembler source used to create an object file for testing readelf's
+   and objdump's ability to process separate debug information files.
+
+   Copyright (C) 2017 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+	
+/* Create a fake .gnu_debuglink section.  */
+
+	.section .gnu_debuglink,"",%progbits
+	.asciz "this_is_a_debuglink.debug"
+	.balign 4
+	.4byte 0x12345678
+
+/* Create a fake .gnu_debugaltlink section.  */
+
+	.section .gnu_debugaltlink,"",%progbits
+	.asciz "linkdebug.debug"
+	.dc.b 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77
+	.dc.b 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff
+	.dc.b 0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef
+   
+/* Create a .debug_str section for local use.  This is also to check
+   the ability to dump the same section twice, if it exists in
+   both the main file and the separate debug info file.  */
+
+	.section	.debug_str,"MS",%progbits,1
+string1:
+	.asciz	"string-1"
+	.asciz  "string-2"
+	.balign	2
+string_end:
+	
+/* Create a .debug_info section that contains strings references into the
+   separate debug info file.  Plus the abbreviations are stored in the
+   separate file too...  */
+
+	.section	.debug_info,"",%progbits
+	.4byte	debugE - debugS	;# Length of Compilation Unit Info
+debugS:
+	.short	0x4	;# DWARF version number.
+	.4byte	0x0	;# Offset into .debug_abbrev section.
+	.byte	0x4	;# Pointer Size (in bytes).
+
+	.uleb128 0x1	;# Use abbrev #1.  This needs a string from the local string table.
+	.4byte	string1
+
+	.uleb128 0x2	;# Use abbrev #2.  This needs a string from the separate string table.
+	.4byte   0x0	;# Avoid complicated expression resolution and hard code the offset...
+
+	;# Minimal section alignment on alpha-* is 2, so ensure no new invalid CU
+	;# will be started.
+	.balign	2, 0
+debugE:
+
diff --git a/binutils/testsuite/binutils-all/dw5.W b/binutils/testsuite/binutils-all/dw5.W
index 977a0c8f16..489fa272de 100644
--- a/binutils/testsuite/binutils-all/dw5.W
+++ b/binutils/testsuite/binutils-all/dw5.W
@@ -342,7 +342,7 @@ Raw dump of debug contents of section .debug_line:
   \[0x00000091\]  Extended opcode 1: End of Sequence
 
 
-Decoded dump of debug contents of section .debug_line:
+Contents of the .debug_line section:
 
 CU: ./main.c:
 File name                            Line number    Starting address    View
diff --git a/binutils/testsuite/binutils-all/linkdebug.s b/binutils/testsuite/binutils-all/linkdebug.s
new file mode 100644
index 0000000000..ff1a1965b2
--- /dev/null
+++ b/binutils/testsuite/binutils-all/linkdebug.s
@@ -0,0 +1,54 @@
+/* Assembler source used to create an object file for testing readelf's
+   and objdump's ability to process separate debug information files.
+
+   Copyright (C) 2017 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+	
+/* This is the separate debug info file.  */
+
+/* Create a .debug_abbrev section for use by the .debug_info section
+   in the main object file.  */
+   
+	.section	.debug_abbrev,"",%progbits
+abbrevs:
+	.uleb128 0x01	;# Abbrev code.
+	.uleb128 0x11	;# DW_TAG_compile_unit
+	.byte	 0x00	;# DW_children_no
+	.uleb128 0x03	;# DW_AT_name
+	.uleb128 0x0e	;# DW_FORM_strp
+	.byte	 0x00	;# End of abbrev
+	.byte	 0x00
+
+	.uleb128 0x02	;# Abbrev code.
+	.uleb128 0x2e	;# DW_TAG_subprogram
+	.byte	 0x00	;# DW_children_no
+	.uleb128 0x03	;# DW_AT_name
+	.uleb128 0x1f21	;# DW_FORM_GNU_strp_alt
+	.byte	 0x0    ;# End of abbrev
+	.byte	 0x0
+
+	.byte	 0x0	;# Abbrevs terminator
+
+/* Create a .debug_str section for remote use.  This is also to check
+   the ability to dump the same section twice, if it exists in
+   both the main file and the separate debug info file.  */
+
+	.section	.debug_str,"MS",%progbits,1
+string3:
+	.asciz	"string-3"
+	.asciz  "string-4"
+	.balign	2
+string_end:
+	
diff --git a/binutils/testsuite/binutils-all/objdump.WK2 b/binutils/testsuite/binutils-all/objdump.WK2
new file mode 100644
index 0000000000..22bcb0dbb0
--- /dev/null
+++ b/binutils/testsuite/binutils-all/objdump.WK2
@@ -0,0 +1,25 @@
+#...
+.*debuglink.o: Found separate debug info file:.*linkdebug.debug
+
+Contents of the .debug_str section \(loaded from .*debuglink.o\):
+
+  0x00000000 73747269 6e672d31 00737472 696e672d string-1.string-
+  0x00000010 3200                                2.
+
+Contents of the .debug_info section \(loaded from .*debuglink.o\):
+
+  Compilation Unit @ offset 0x0:
+   Length:        0x12 \(32-bit\)
+   Version:       4
+   Abbrev Offset: 0x0
+   Pointer Size:  4
+ <0><b>: Abbrev Number: 1 \(DW_TAG_compile_unit\)
+    <c>   DW_AT_name        : \(indirect string, offset: 0x0\): string-1
+ <0><10>: Abbrev Number: 2 \(DW_TAG_subprogram\)
+    <11>   DW_AT_name        : \(alt indirect string, offset: 0x0\) string-3
+
+Contents of the .debug_str section \(loaded from .*linkdebug.debug\):
+
+  0x00000000 73747269 6e672d33 00737472 696e672d string-3.string-
+  0x00000010 3400                                4.
+
diff --git a/binutils/testsuite/binutils-all/objdump.WL b/binutils/testsuite/binutils-all/objdump.WL
index 7c5a591a6d..c8bac334e5 100644
--- a/binutils/testsuite/binutils-all/objdump.WL
+++ b/binutils/testsuite/binutils-all/objdump.WL
@@ -1,7 +1,7 @@
 
 .*dw2-decodedline.o:     file format .*
 
-Decoded dump of debug contents of section \.debug_line:
+Contents of the \.debug_line section:
 
 CU: \./dw2-decodedline\.c:
 File name                            Line number    Starting address    View
diff --git a/binutils/testsuite/binutils-all/objdump.Wk b/binutils/testsuite/binutils-all/objdump.Wk
new file mode 100644
index 0000000000..35472328fc
--- /dev/null
+++ b/binutils/testsuite/binutils-all/objdump.Wk
@@ -0,0 +1,8 @@
+tmpdir/debuglink\.o:     file format .*
+Contents of the \.gnu_debuglink section:
+  Separate debug info file: this_is_a_debuglink\.debug
+  CRC value: 0x12345678
+Contents of the \.gnu_debugaltlink section:
+  Separate debug info file: linkdebug\.debug
+  Build-ID \(0x18 bytes\):
+ 00 11 22 33 44 55 66 77 88 99 aa bb cc dd ee ff 01 23 45 67 89 ab cd ef
diff --git a/binutils/testsuite/binutils-all/objdump.exp b/binutils/testsuite/binutils-all/objdump.exp
index 47cda4562d..8ee350ff89 100644
--- a/binutils/testsuite/binutils-all/objdump.exp
+++ b/binutils/testsuite/binutils-all/objdump.exp
@@ -424,6 +424,89 @@ if {[isnative] && [is_elf_format]} then {
     test_build_id_debuglink
 }
 
+# Test objdump -Wk on a file containing debug links.
+
+if { [is_elf_format] } then {
+    set testsrc $srcdir/$subdir/debuglink.s
+
+    if { ![binutils_assemble $testsrc tmpdir/debuglink.o] } then {
+	fail "objdump -Wk (reason: could not assemble source)"
+    }
+
+    if [is_remote host] {
+	set testfile [remote_download host tmpdir/debuglink.o]
+    } else {
+	set testfile tmpdir/debuglink.o
+    }
+
+    set got [remote_exec host "$OBJDUMP $OBJDUMPFLAGS -Wk $testfile" "" "/dev/null" "objdump.out"]
+
+    if { [lindex $got 0] != 0 || ![string match "" [lindex $got 1]] } then {
+	fail "objdump -Wk (reason: unexpected output)"
+	send_log $got
+	send_log "\n"
+    }
+
+    if { [regexp_diff objdump.out $srcdir/$subdir/objdump.Wk] } then {
+	fail "objdump -Wk"
+    } else {
+	pass "objdump -Wk"
+    }
+}
+
+
+# Very similar to proc test_build_id_debuglink except this time we
+# display some of the contents of the separate debug info file.
+
+proc test_follow_debuglink {} {
+    global srcdir
+    global subdir
+    global OBJDUMP
+    
+    set test "follow-debuglink"
+
+    if {![binutils_assemble $srcdir/$subdir/debuglink.s tmpdir/debuglink.o]} then {
+	fail "$test (reason: assemble first source file)"
+	return
+    }
+
+    if {![binutils_assemble $srcdir/$subdir/linkdebug.s tmpdir/linkdebug.debug]} then {
+	fail "$test (reason: assemble second source file)"
+	return
+    }
+
+    if ![is_remote host] {
+	set tempfile tmpdir/debuglink.o
+    } else {
+	set tempfile [remote_download host tmpdir/linkdebug.debug]
+	set tempfile [remote_download host tmpdir/debuglink.o]
+    }
+    
+    set got [remote_exec host "$OBJDUMP --dwarf=follow-links --dwarf=info --dwarf=str $tempfile" "" "/dev/null" "tmpdir/objdump.out"]
+    if { [lindex $got 0] != 0 || ![string match "" [lindex $got 1]] } then {
+	fail "$test (reason: unexpected error output from objdump)"
+	return
+    }
+
+    if { [regexp_diff tmpdir/objdump.out $srcdir/$subdir/objdump.WK2] } then {
+	fail $test
+	verbose "output is \n[file_contents objdump.out]" 2
+	return
+    }
+
+    pass $test
+
+    # Tidy up
+    set got [remote_exec host "rm tmpdir/objdump.out"]
+    set got [remote_exec host "rm $tempfile"]
+    set got [remote_exec host "rm tmpdir/linkdebug.debug"]
+}
+
+if {[isnative] && [is_elf_format]} then {
+    test_follow_debuglink
+}
+
+
 # Options which are not tested: -a -d -D -R -T -x -l --stabs
 # I don't see any generic way to test any of these other than -a.
 # Tests could be written for specific targets, and that should be done
diff --git a/binutils/testsuite/binutils-all/readelf.exp b/binutils/testsuite/binutils-all/readelf.exp
index 10fd8d8da7..fdb310ca19 100644
--- a/binutils/testsuite/binutils-all/readelf.exp
+++ b/binutils/testsuite/binutils-all/readelf.exp
@@ -466,3 +466,35 @@ if {![binutils_assemble_flags $srcdir/$subdir/dwarf-attributes.S tmpdir/dwarf-at
     # Make sure that readelf can decode the contents.
     readelf_test -wi $tempfile dwarf-attributes.W {}
 }
+
+# Check that debug link sections can be dumped.
+if {![binutils_assemble $srcdir/$subdir/debuglink.s tmpdir/debuglink.o]} then {
+    unresolved "readelf --debug-dump=links (failed to assemble)"
+} else {
+    if ![is_remote host] {
+	set tempfile tmpdir/debuglink.o
+    } else {
+	set tempfile [remote_download host tmpdir/debuglink.o]
+    }
+
+    readelf_test {--debug-dump=links} $tempfile readelf.k  {}
+}
+
+# Check that debug link sections can be followed.
+if {![binutils_assemble $srcdir/$subdir/debuglink.s tmpdir/debuglink.o]} then {
+    unresolved "readelf --debug-dump=follow-links (failed to assemble1)"
+} else {
+    if {![binutils_assemble $srcdir/$subdir/linkdebug.s tmpdir/linkdebug.debug]} then {
+	unresolved "readelf --debug-dump=follow-links (failed to assemble2)"
+    } else {
+	if ![is_remote host] {
+	    set tempfile tmpdir/debuglink.o
+	} else {
+	    set tempfile [remote_download host tmpdir/linkdebug.debug]
+	    set tempfile [remote_download host tmpdir/debuglink.o]
+	}
+
+	readelf_test {-wKis} $tempfile objdump.WK2  {}
+    }
+}
+
diff --git a/binutils/testsuite/binutils-all/readelf.k b/binutils/testsuite/binutils-all/readelf.k
new file mode 100644
index 0000000000..fe03342118
--- /dev/null
+++ b/binutils/testsuite/binutils-all/readelf.k
@@ -0,0 +1,7 @@
+Contents of the \.gnu_debuglink section:
+  Separate debug info file: this_is_a_debuglink\.debug
+  CRC value: 0x12345678
+Contents of the \.gnu_debugaltlink section:
+  Separate debug info file: linkdebug\.debug
+  Build-ID \(0x18 bytes\):
+ 00 11 22 33 44 55 66 77 88 99 aa bb cc dd ee ff 01 23 45 67 89 ab cd ef