DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on November 03, 2025 has been entered.
In view of the amendment to the claims, the amendment of claims 9, 17, 22-23 and 29 have been acknowledged.
In view of the amendment of claims 9 and 17, Applicant amended each claim to describe “output data based on information in association with position information in association with a position in an image of the multiple images”. In additional, Applicant provided paragraphs of the specification corresponding to the features recited in claims 25, 28, 40, 42. Accordingly, the 35 U.S.C. 112 rejections of claims 9, 25, 28-29, 33-34, 40 and 42 have been withdrawn.
Response to Arguments
Applicant’s arguments, see pages 9-12 of Remarks, filed November 03, 2025 have been fully considered.
Regarding to the 35 U.S.C. 112 rejections corresponding to pages 9-10 of Remarks.
Examiner replies:
As discussed above, the 35 U.S.C. 112 rejections of claims 9, 25, 28-29, 33-34, 40 and 42 have been withdrawn.
Regarding Claims 41 and 42, Applicants state in pages 11-12 of Remarks that “In regard to independent claims 41 and 42, Applicant respectfully submits that Heinen fails to teach or suggest "outputting data based on information in association with a position information in association with a position in an image of the multiple images, the data including link information," as claimed. The Office Action rejects claims 41 and 42 based on Heinen that discloses "timeline system 120104" (see paragraph [0191], FIG. 2u, FIG. 2v, and FIGS. 20 to 2q). The Office Action asserts that "timeline system 120104" corresponds to "multiple images respectively corresponding to viewpoints" in Applicant's claims 41 and 42.
... ...
Therefore, Heinen does not disclose the configuration of "information in association with a position information in association with a position in an image of the multiple images" as claimed in claims 41 and 42. Thus, Applicant respectfully submits that Heinen fails to teach or suggest "receiving an input of a designation of the link information; and displaying, on the display, the image among the multiple images, in accordance with the designation of the link information" as claimed in claim 41 and "receiving an input of a designation of the information; and displaying, on the display, the image among the multiple images, in accordance with designated information" as claimed in claim 42. Independent claims 9 and 17 (and all claims depending thereon) are patentable for similar reasons.
...”.
Examiner replies:
The examiner disagrees with Applicant’s premises and conclusion. Claim 41 recites “displaying, on a display, at least one image by switching between multiple images respectively corresponding to viewpoints; outputting data based on information in association with a position information in association with a position in an image of the multiple images, the data including link information; receiving an input of a designation of the link information; and displaying, on the display, the image among the multiple images, in accordance with the designation of the link information”. Thus, the claim just simply recite “information in association with a position information in association with a position in an image”. Under the broadest reasonable interpretation, the examiner respectfully maintains that the prior art rejections in this case are proper for the following reasons. In respond to the appellant arguments, the examiner recites Heinen in order to disclose the issue. Heinen discloses a system and a method for importing, creating and editing the timeline of 360° images and videos (As shown in FIGS. 2a-2y). For example, as shown in FIG. 2i and paragraph [0183], Heinen describes “the user 120110 may be able to add and edit 2D as well as 3D content 120914 in the uploaded 360° image or video through the UI”. Thus, each added or edited object “129014” indicates a position in an image of timeline images. Therefore, the timeline information includes a time-based scene and a position information of object in the image of timeline of 360° images and videos. The examiner cited paragraphs [0211]-[0213] of Heinen in the previous office action. Those paragraphs describe multiple images are taken with cameras. As discussed above, Heinen discloses a system and a method for importing, creating and editing the timeline of 360° images and videos. In additional, the paragraph [0191] of Heinen describes that the viewer selects the appropriate timeline-element to display an image among the timeline of 360° images and videos. Accordingly, Heinen discloses the features recited in claim 41; more specifically the features of “outputting data based on information in association with a position information in association with a position in an image of the multiple images, the data including link information; receiving an input of a designation of the link information; and displaying, on the display, the image among the multiple images, in accordance with the designation of the link information” recited in claim 41 and the above arguments.
Independent claims 9, 17 and 42 recite the features similar to claim 41 and have the same above reasons.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 9, 17-20, 37-38 and 41-42 are rejected under 35 U.S.C. 102 (a)(2) as being anticipated by HEINEN et al (U.S. Patent Application Publication 20200312029 A1).
Regarding claim 9, HEINEN discloses an image processing apparatus (Paragraph [0368], The present embodiments can be embodied in the form of methods and apparatus for practicing those methods) comprising:
circuitry (Paragraph [0368], the present embodiments can also be embodied in the form of program code embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the embodiments ... When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits) configured to:
display, on a display (Paragraph [0194], FIG. 2i to FIG. 2y illustrate example user interface(s) embodiment(s) of the methods and systems described here ... While the interfaces described above were designed for desktop computers and browsers, the methods and systems here can be transferred into the context of devices with different input methods such as head-mounted displays that use hands-free interaction (e.g., pointing a cross hair by moving your head) or mobile touch devices such as smartphones, laptops and tablets with touch screen interfaces), at least one image by switching between multiple images respectively (Paragraph [0191], FIG. 2u and FIG. 2v depict example embodiments of an expanded timeline 122102 as visualization of the timeline system 120104 through which the viewer 120118 can follow the changes in the 360° images or videos 122002, 122202 of a certain location over a period of time; paragraph [0183], FIG. 2i illustrates an example combined user interface (UI) of the timeline system 120104 and the existing system 120120 “As shown in FIG. 2a” as a VR editor for creating and editing VR tours with 360° images and videos 120902 with closed timeline panel ... Each location-based scene may be listed in the scene overview 120916 as scene previews. To create a virtual reality tour for a specified area, the user 120110 may be able to add multiple locations 120912 as location-based scenes 120906 120908. Each location represented as location-based scene 120906 120908 can hold its' own timeline; paragraphs [0184]-[0190] describe timeline panel of the timeline system for each specific location-based scene) corresponding to viewpoints (As shown in FIGS. 2o to 2q, various locations of rooms correspond to each specific location-based scene in the timeline. Thus, each location-based scene listed in the scene overview 120916 corresponding to multiple viewpoints of rooms);
output data based on information (Paragraph [0191], FIG. 2u shows an expanded timeline 122102 as visualization of the timeline system 120104 through which the viewer 120118 can follow the changes in the 360° images or videos 122002, 122202 of a certain location over a period of time. The timeline example 122102 has one location-based scene 122104 and at least one time-based scene 122106, 122108, 122110, 122112; FIG. 2i and paragraph [0183], the user 120110 may be able to add and edit 2D as well as 3D content 120914 in the uploaded 360° image or video through the UI. Thus, each added or edited object “129014” indicates a position in an image of timeline images) in association with position information in association with a position in an image of the multiple images (Paragraph [0211], FIG. 5a illustrates an exemplary way 150100 to apply a position to a photo in a scene on the system, relative to an imported floor plan ... When the user takes a photo 150106 with the camera 150102, 400108, he can also interact 150112 with the document 150110 on the device 150108, 400106, for example by pressing on the screen. With this interaction 150112, the user can choose the appropriate location on the example document 150110 where the image was taken with the camera 150102, 400108; paragraph [0212], ... After one or multiple photos have been taken and one or multiple locations 150202 have been chosen, one or multiple locations 150202 could be visualized on a device 150108, 400106 on a map 150204 or other display such as a floorplan, at a place where the user has chosen the locations 150202 ...; paragraph [0213], FIG. 5c is an example flowchart that illustrates a use case where a user 150302 regularly revisits locations in the real world which are depicted in the floor plan, to take photos at various points in time of these locations, for example weekly photos of every room in a building where the walking tour could be the same every week. When a user 150302 takes a photo 150304 with a camera 400108, in the example, the photo can be stored together with a timestamp 150306 ...);
receive an input of a designation of the data (Paragraph [0191], the viewer 120118 selects the appropriate timeline-element 122110. As shown in FIG. 2u, the finger touches the timeline to select time-based scene 122110); and
display, on the display, the image corresponding to a viewpoint that corresponds to the designation of the data (Paragraph [0191], selecting the appropriate timeline-element 122110 displaying the changed scene 122202 of the location at a subsequent time in contrast to the original scene 122002, as shown in FIG. 2v. As shown in FIG. 2v, the scene 122202 displays an image of location-based scene corresponding to a viewpoint within a room), from among the multiple images (Paragraph [0191], FIG. 2u shows timeline 122102 including time-based scene 122104, 122106, 122108, 122110, 122112. As discussed above, each time-based scene represents various locations of a room corresponding to each viewpoint).
Regarding claim 17, HEINEN discloses a non-transitory computer-readable storage medium storing executable instructions, which when executed by circuitry, cause the circuitry to perform a method (Paragraph [0368], the present embodiments can also be embodied in the form of program code embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the embodiments ... When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits), the method comprising:
displaying, on a display (Paragraph [0194], FIG. 2i to FIG. 2y illustrate example user interface(s) embodiment(s) of the methods and systems described here ... While the interfaces described above were designed for desktop computers and browsers, the methods and systems here can be transferred into the context of devices with different input methods such as head-mounted displays that use hands-free interaction (e.g., pointing a cross hair by moving your head) or mobile touch devices such as smartphones, laptops and tablets with touch screen interfaces), at least one image by switching between multiple images respectively (Paragraph [0191], FIG. 2u and FIG. 2v depict example embodiments of an expanded timeline 122102 as visualization of the timeline system 120104 through which the viewer 120118 can follow the changes in the 360° images or videos 122002, 122202 of a certain location over a period of time; paragraph [0183], FIG. 2i illustrates an example combined user interface (UI) of the timeline system 120104 and the existing system 120120 “As shown in FIG. 2a” as a VR editor for creating and editing VR tours with 360° images and videos 120902 with closed timeline panel ... Each location-based scene may be listed in the scene overview 120916 as scene previews. To create a virtual reality tour for a specified area, the user 120110 may be able to add multiple locations 120912 as location-based scenes 120906 120908. Each location represented as location-based scene 120906 120908 can hold its' own timeline; paragraphs [0184]-[0190] describe timeline panel of the timeline system for each specific location-based scene) corresponding to viewpoints (As shown in FIGS. 2o to 2q, various locations of rooms correspond to each specific location-based scene in the timeline. Thus, each location-based scene listed in the scene overview 120916 corresponding to multiple viewpoints of rooms);
outputting data based on information (Paragraph [0191], FIG. 2u shows an expanded timeline 122102 as visualization of the timeline system 120104 through which the viewer 120118 can follow the changes in the 360° images or videos 122002, 122202 of a certain location over a period of time. The timeline example 122102 has one location-based scene 122104 and at least one time-based scene 122106, 122108, 122110, 122112; FIG. 2i and paragraph [0183], the user 120110 may be able to add and edit 2D as well as 3D content 120914 in the uploaded 360° image or video through the UI. Thus, each added or edited object “129014” indicates a position in an image of timeline images) in association with position information in association with a position in an image of the multiple images (Paragraph [0211], FIG. 5a illustrates an exemplary way 150100 to apply a position to a photo in a scene on the system, relative to an imported floor plan ... When the user takes a photo 150106 with the camera 150102, 400108, he can also interact 150112 with the document 150110 on the device 150108, 400106, for example by pressing on the screen. With this interaction 150112, the user can choose the appropriate location on the example document 150110 where the image was taken with the camera 150102, 400108; paragraph [0212], ... After one or multiple photos have been taken and one or multiple locations 150202 have been chosen, one or multiple locations 150202 could be visualized on a device 150108, 400106 on a map 150204 or other display such as a floorplan, at a place where the user has chosen the locations 150202 ...; paragraph [0213], FIG. 5c is an example flowchart that illustrates a use case where a user 150302 regularly revisits locations in the real world which are depicted in the floor plan, to take photos at various points in time of these locations, for example weekly photos of every room in a building where the walking tour could be the same every week. When a user 150302 takes a photo 150304 with a camera 400108, in the example, the photo can be stored together with a timestamp 150306 ...);
receiving an input of a designation of the data (Paragraph [0191], the viewer 120118 selects the appropriate timeline-element 122110. As shown in FIG. 2u, the finger touches the timeline to select time-based scene 122110); and
displaying, on the display, the image corresponding to a viewpoint that corresponds to the designation of the data (Paragraph [0191], selecting the appropriate timeline-element 122110 displaying the changed scene 122202 of the location at a subsequent time in contrast to the original scene 122002, as shown in FIG. 2v. As shown in FIG. 2v, the scene 122202 displays an image of location-based scene corresponding to a viewpoint within a room), from among the multiple images (Paragraph [0191], FIG. 2u shows timeline 122102 including time-based scene 122104, 122106, 122108, 122110, 122112. As discussed above, each time-based scene represents various locations of a room corresponding to each viewpoint).
Regarding claim 18, HEINEN discloses everything claimed as applied above (see claim 17), and HEINEN discloses further comprising:
displaying a first screen (Paragraph [0207], FIG. 3g shows a user interface for Hotspot navigation; the display window under floor plan 130702 includes 130508 and 130510 sections, those two sections can be interpreted as a first screen) and a second screen on the display (Paragraph [0207], an active Floor Plan 130512 as seen in FIG. 3e can be enlarged by the user; the section of an enlarged Floor Plan 130702 can be interpreted as a second screen), the first screen is to display the multiple images at different times with multiple viewpoints by switching between the multiple images (Paragraph [0207], the section 130508 displays a Scene List 130508 ... If the active Scene 130502 is interconnected to a Hotspot 130704 ... Hotspots 130706 interconnected to non-active Scenes 130504; paragraph [0205], the central section 130510 may display the content of the currently active Scene 130502. Thus, the first screen display multiple scene images with multiple viewpoints of room and the user select Hotspot on the enlarged Floor Plan 130702 to switch active Scene between scene1, scene2 ... Then, the content of the currently active Scene is displayed in the central section 130510), and the second screen is to display a floor plan image of a space (Paragraph [0207], the enlarged Floor Plan 130702 may be overlaid (similar to the smaller Floor Plan representation 130512 in FIG. 3e) by the interconnected Hotspots 130704 130706 with different colors and/or icons) by superimposing multiple markers respectively corresponding to the multiple viewpoints on the floor plan image (Paragraph [0197], Holos may be created or imported to show the various rooms; Paragraph [0203], FIG. 3c illustrates an example additional or alternative process of interconnecting a selected Scene with a location on an imported Floor Plan using a technical device 400106 ... Using the enlarged Floor Plan a location to which the selected Scene should be interconnected to can be pointed out on the Floor Plan 130306. The interconnection between the Scene and a location on the selected Floor Plan may form a Hotspot on the Floor Plan ...; paragraph [0207], if the active Scene 130502 is interconnected to a Hotspot 130704, this Hotspot 130704 may be highlighted compared to Hotspots 130706 interconnected to non-active Scenes 130504);
receiving a selection of one marker from the multiple markers on the second screen via the second screen (FIG. 3g; paragraph [0207], the user may rearrange a Hotspot 130704 for the active Scene 130502 by selecting the desired location on the enlarged Floor Plan 130702. Rearranging an existing Hotspot can be done by the user with drag-and-drop or other interaction with the UI); and
displaying, on the first screen, an image with a viewpoint corresponding to the selected one marker to display the first screen on the display (Paragraph [0207], the user may rearrange a Hotspot 130704 for the active Scene 130502 by selecting the desired location on the enlarged Floor Plan 130702 ... A selection of an existing Hotspot may navigate the user to the interconnected Scene; FIG. 3e; paragraph [0205], the central section 130510 may display the content of the currently active Scene 130502).
Regarding claim 19, HEINEN discloses everything claimed as applied above (see claim 17), and HEINEN discloses further comprising:
receiving an instruction to move from a first viewpoint to a second viewpoint on a second screen (Paragraph [0207], an active Floor Plan 130512 as seen in FIG. 3e can be enlarged by the user; the section of an enlarged Floor Plan 130702 can be interpreted as a second screen; paragraph [0203], FIG. 3c illustrates an example additional or alternative process of interconnecting a selected Scene with a location on an imported Floor Plan using a technical device 400106 ... Using the enlarged Floor Plan a location to which the selected Scene should be interconnected to can be pointed out on the Floor Plan 130306. The interconnection between the Scene and a location on the selected Floor Plan may form a Hotspot on the Floor Plan ...; paragraph [0207], if the active Scene 130502 is interconnected to a Hotspot 130704, this Hotspot 130704 may be highlighted compared to Hotspots 130706 interconnected to non-active Scenes 130504), with a first screen (Paragraph [0207], FIG. 3g shows a user interface for Hotspot navigation; the display window under floor plan 130702 includes 130508 and 130510 sections, those two sections can be interpreted as a first screen) displaying a first image with the first viewpoint (Paragraph [0207], the section 130508 displays a Scene List 130508 ... If the active Scene 130502 is interconnected to a Hotspot 130704 ...; paragraph [0205], the central section 130510 may display the content of the currently active Scene 130502); and
switching from the first image with the first viewpoint to a second image with the second viewpoint to display the second image with the second viewpoint on the first screen (Paragraph [0207], the user may rearrange a Hotspot 130704 for the active Scene 130502 by selecting the desired location on the enlarged Floor Plan 130702 ... A selection of an existing Hotspot may navigate the user to the interconnected Scene; FIG. 3e; paragraph [0205], the central section 130510 may display the content of the currently active Scene 130502), in response to the received instruction (FIG. 3g; paragraph [0207], the user may rearrange a Hotspot 130704 for the active Scene 130502 by selecting the desired location on the enlarged Floor Plan 130702. Rearranging an existing Hotspot can be done by the user with drag-and-drop or other interaction with the UI).
Regarding claim 20, HEINEN discloses everything claimed as applied above (see claim 17), and HEINEN further discloses wherein a first image with a first viewpoint is an image captured from the first viewpoint at an angle of view of 360 degrees (FIG. 5a; paragraph [0211], the system in the example comprises a camera 150102, 390102, including a fully 360° spherical camera; FIG. 5d; paragraph [0214], after a photo 150402 is taken, including full 360° spherical photos), and
a second image with a second viewpoint is an image captured from the second viewpoint at an angle of view of 360 degrees (FIG. 5c; paragraph [0213], a use case where a user 150302 regularly revisits locations in the real world which are depicted in the floor plan, to take photos at various points in time of these locations ... When a user 150302 takes a photo 150304 with a camera 400108, in the example, the photo can be stored together with a timestamp 150306. The user 150302 can additionally or alternatively store a corresponding location on another device 400106. The user 150302 can either choose a new location 150308 ... Thus, the second photo is captured at the new location; paragraph [0214], after a photo 150402 is taken, including full 360° spherical photos).
Regarding claim 37, HEINEN discloses everything claimed as applied above (see claim 17), and HEINEN discloses further comprising:
displaying multiple pieces of information (Paragraph [0191], FIG. 2u shows an expanded timeline 122102 as visualization of the timeline system 120104 through which the viewer 120118 can follow the changes in the 360° images or videos 122002, 122202 of a certain location over a period of time. The timeline example 122102 has one location-based scene 122104 and at least one time-based scene 122106, 122108, 122110, 122112), and
receiving a designation of one of the multiple pieces of information (Paragraph [0191], the viewer 120118 selects the appropriate timeline-element 122110. As shown in FIG. 2u, the finger touches the timeline to select time-based scene 122110).
Regarding claim 38, HEINEN discloses everything claimed as applied above (see claim 17), and HEINEN discloses wherein the information further includes link information (Paragraph [0174], FIG. 2a, the user 120110 could either work with an existing system 120120, which uses the systems here 120104 and its interfaces to create and output content ...; paragraph [0184], FIG. 2j shows a possible visualization of the expanded timeline panel 121002 of the timeline system 120104 in an exemplary embedding system 120120 as introduced in FIG. 2i. A timeline for a specific location-based scene 120906 may hold at least two time-based scenes as timeline-elements 121004, 121006, 121008 with one of the scenes being the location-based scene 121004 marking time t0. Each timeline-element 121004 121006, 121008 may have a date and time and are sorted chronologically by date in ascending order. Thus, each timeline-element 121004 121006, 121008 provides link information with one of the scenes), and the method further includes receiving a designation of the link information (Paragraph [0191], the viewer 120118 selects the appropriate timeline-element 122110. As shown in FIG. 2u, the finger touches the timeline to select time-based scene 122110) and displaying the image in accordance with the link information (Paragraph [0191], selecting the appropriate timeline-element 122110 displaying the changed scene 122202 of the location at a subsequent time in contrast to the original scene 122002, as shown in FIG. 2v. As shown in FIG. 2v, the scene 122202 displays an image of location-based scene corresponding to a viewpoint within a room).
Regarding claim 41, HEINEN discloses a non-transitory computer-readable storage medium storing executable instructions, which when executed by circuitry, cause the circuitry to perform a method (Paragraph [0368], the present embodiments can also be embodied in the form of program code embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the embodiments ... When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits), the method comprising:
displaying, on a display (Paragraph [0194], FIG. 2i to FIG. 2y illustrate example user interface(s) embodiment(s) of the methods and systems described here ... While the interfaces described above were designed for desktop computers and browsers, the methods and systems here can be transferred into the context of devices with different input methods such as head-mounted displays that use hands-free interaction (e.g., pointing a cross hair by moving your head) or mobile touch devices such as smartphones, laptops and tablets with touch screen interfaces), at least one image by switching between multiple images respectively (Paragraph [0191], FIG. 2u and FIG. 2v depict example embodiments of an expanded timeline 122102 as visualization of the timeline system 120104 through which the viewer 120118 can follow the changes in the 360° images or videos 122002, 122202 of a certain location over a period of time; paragraph [0183], FIG. 2i illustrates an example combined user interface (UI) of the timeline system 120104 and the existing system 120120 “As shown in FIG. 2a” as a VR editor for creating and editing VR tours with 360° images and videos 120902 with closed timeline panel ... Each location-based scene may be listed in the scene overview 120916 as scene previews. To create a virtual reality tour for a specified area, the user 120110 may be able to add multiple locations 120912 as location-based scenes 120906 120908. Each location represented as location-based scene 120906 120908 can hold its' own timeline; paragraphs [0184]-[0190] describe timeline panel of the timeline system for each specific location-based scene) corresponding to viewpoints (As shown in FIGS. 2o to 2q, various locations of rooms correspond to each specific location-based scene in the timeline. Thus, each location-based scene listed in the scene overview 120916 corresponding to multiple viewpoints of rooms.);
outputting data based on information (Paragraph [0191], FIG. 2u shows an expanded timeline 122102 as visualization of the timeline system 120104 through which the viewer 120118 can follow the changes in the 360° images or videos 122002, 122202 of a certain location over a period of time. The timeline example 122102 has one location-based scene 122104 and at least one time-based scene 122106, 122108, 122110, 122112) in association with a position information in association with a position in an image of the multiple images (Paragraph [0211], FIG. 5a illustrates an exemplary way 150100 to apply a position to a photo in a scene on the system, relative to an imported floor plan ... When the user takes a photo 150106 with the camera 150102, 400108, he can also interact 150112 with the document 150110 on the device 150108, 400106, for example by pressing on the screen. With this interaction 150112, the user can choose the appropriate location on the example document 150110 where the image was taken with the camera 150102, 400108; paragraph [0212], ... After one or multiple photos have been taken and one or multiple locations 150202 have been chosen, one or multiple locations 150202 could be visualized on a device 150108, 400106 on a map 150204 or other display such as a floorplan, at a place where the user has chosen the locations 150202 ...; paragraph [0213], FIG. 5c is an example flowchart that illustrates a use case where a user 150302 regularly revisits locations in the real world which are depicted in the floor plan, to take photos at various points in time of these locations, for example weekly photos of every room in a building where the walking tour could be the same every week. When a user 150302 takes a photo 150304 with a camera 400108, in the example, the photo can be stored together with a timestamp 150306 ...), the data including link information (Paragraph [0174], FIG. 2a, the user 120110 could either work with an existing system 120120, which uses the systems here 120104 and its interfaces to create and output content ...; paragraph [0184], FIG. 2j shows a possible visualization of the expanded timeline panel 121002 of the timeline system 120104 in an exemplary embedding system 120120 as introduced in FIG. 2i. A timeline for a specific location-based scene 120906 may hold at least two time-based scenes as timeline-elements 121004, 121006, 121008 with one of the scenes being the location-based scene 121004 marking time t0. Each timeline-element 121004 121006, 121008 may have a date and time and are sorted chronologically by date in ascending order. Thus, each timeline-element 121004 121006, 121008 provides link information with one of the scenes);
receiving an input of a designation of the link information (Paragraph [0191], the viewer 120118 selects the appropriate timeline-element 122110. As shown in FIG. 2u, the finger touches the timeline to select time-based scene 122110); and
displaying, on the display, the image among the multiple images, in accordance with the designation of the link information (Paragraph [0191], selecting the appropriate timeline-element 122110 displaying the changed scene 122202 of the location at a subsequent time in contrast to the original scene 122002, as shown in FIG. 2v. As shown in FIG. 2v, the scene 122202 displays an image of location-based scene corresponding to a viewpoint within a room).
Regarding claim 42, HEINEN discloses a non-transitory computer-readable storage medium storing executable instructions, which when executed by circuitry, cause the circuitry to perform a method (Paragraph [0368], the present embodiments can also be embodied in the form of program code embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the embodiments ... When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits), the method comprising:
display, on a display (Paragraph [0194], FIG. 2i to FIG. 2y illustrate example user interface(s) embodiment(s) of the methods and systems described here ... While the interfaces described above were designed for desktop computers and browsers, the methods and systems here can be transferred into the context of devices with different input methods such as head-mounted displays that use hands-free interaction (e.g., pointing a cross hair by moving your head) or mobile touch devices such as smartphones, laptops and tablets with touch screen interfaces), at least one image by switching between multiple images respectively (Paragraph [0191], FIG. 2u and FIG. 2v depict example embodiments of an expanded timeline 122102 as visualization of the timeline system 120104 through which the viewer 120118 can follow the changes in the 360° images or videos 122002, 122202 of a certain location over a period of time; paragraph [0183], FIG. 2i illustrates an example combined user interface (UI) of the timeline system 120104 and the existing system 120120 “As shown in FIG. 2a” as a VR editor for creating and editing VR tours with 360° images and videos 120902 with closed timeline panel ... Each location-based scene may be listed in the scene overview 120916 as scene previews. To create a virtual reality tour for a specified area, the user 120110 may be able to add multiple locations 120912 as location-based scenes 120906 120908. Each location represented as location-based scene 120906 120908 can hold its' own timeline; paragraphs [0184]-[0190] describe timeline panel of the timeline system for each specific location-based scene) corresponding to viewpoints (As shown in FIGS. 2o to 2q, various locations of rooms correspond to each specific location-based scene in the timeline. Thus, each location-based scene listed in the scene overview 120916 corresponding to multiple viewpoints of rooms);
displaying information (Paragraph [0191], FIG. 2u shows an expanded timeline 122102 as visualization of the timeline system 120104 through which the viewer 120118 can follow the changes in the 360° images or videos 122002, 122202 of a certain location over a period of time. The timeline example 122102 has one location-based scene 122104 and at least one time-based scene 122106, 122108, 122110, 122112) in association with a position information in association with a position in an image of the multiple images (Paragraph [0211], FIG. 5a illustrates an exemplary way 150100 to apply a position to a photo in a scene on the system, relative to an imported floor plan ... When the user takes a photo 150106 with the camera 150102, 400108, he can also interact 150112 with the document 150110 on the device 150108, 400106, for example by pressing on the screen. With this interaction 150112, the user can choose the appropriate location on the example document 150110 where the image was taken with the camera 150102, 400108; paragraph [0212], ... After one or multiple photos have been taken and one or multiple locations 150202 have been chosen, one or multiple location 150202 could be visualized on a device 150108, 400106 on a map 150204 or other display such as a floorplan, at a place where the user has chosen the locations 150202 ...; paragraph [0213], FIG. 5c is an example flowchart that illustrates a use case where a user 150302 regularly revisits locations in the real world which are depicted in the floor plan, to take photos at various points in time of these locations, for example weekly photos of every room in a building where the walking tour could be the same every week. When a user 150302 takes a photo 150304 with a camera 400108, in the example, the photo can be stored together with a timestamp 150306 ...);
receiving an input of a designation of the information (Paragraph [0191], the viewer 120118 selects the appropriate timeline-element 122110. As shown in FIG. 2u, the finger touches the timeline to select time-based scene 122110); and
displaying, on the display, the image among the multiple images, in accordance with designated information (Paragraph [0191], selecting the appropriate timeline-element 122110 displaying the changed scene 122202 of the location at a subsequent time in contrast to the original scene 122002, as shown in FIG. 2v. As shown in FIG. 2v, the scene 122202 displays an image of location-based scene corresponding to a viewpoint within a room).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over HEINEN et al (U.S. Patent Application Publication 20200312029 A1) in view of Lynch (U.S. Patent Application Publication 20140002440 A1).
Regarding claim 25, HEINEN discloses everything claimed as applied above (see claim 17).
However, HEINEN does not specifically disclose wherein the information includes a cut image obtained from the multiple images.
In additional, Lynch discloses (Abstract, in one embodiment, a building model is presented to a user. The building model may be a three-dimensional rendering of a geographic region. A user selects a point on the three-dimensional rendering of a geographic region. Based on the selected point one or more image bubbles are chosen ...; paragraph [0024], the disclosed embodiments relate to a method and system for presenting street side imagery in combination with a rendering of a geographical area. Street side imagery includes photographic images or images derived from photographic images. The street side imagery may be collected from a camera at street level. The rendering of a geographic area is a model or representation of a portion of the Earth and objects on the surface of the Earth ...) wherein the information includes a cut image obtained from the multiple images (Paragraph [0046], FIG. 5 illustrates another exemplary perspective view 160 including street side imagery 161. The location of the street side imagery 161 in the perspective view 160 also corresponds to the location of the subject matter. Street side imagery 161 is the top of the building as seen from the street level elevation. FIGS. 4 and 5 illustrates that the street side imagery may include subject matter at different elevations even when the street side imagery is collected at the same elevation. In this case, the vector from the image bubble center to the selected building location angles steeply upward. Therefore, the extracted portion of the image bubble is taken from the upper part of the image bubble instead if a side facing section).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the systems for creating a virtual reality scene taught by HEINEN incorporate the teachings of Lynch, applying the system for presenting street side imagery taught by Lynch to allow the user select a point on the image; then extract the portion of the image based on the user selection and provide the cut image in the navigation interface. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify HEINEN according to the relied-upon teachings of Lynch to obtain the invention as specified in claim.
Claims 27 and 33-36 are rejected under 35 U.S.C. 103 as being unpatentable over HEINEN et al (U.S. Patent Application Publication 20200312029 A1) in view of Laughlin et al (U.S. Patent No. 10,107,767 B1).
Regarding claim 27, HEINEN discloses everything claimed as applied above (see claim 17).
However, HEINEN does not specifically disclose wherein position information is input in accordance with a designation of a position on the image.
In additional, Laughlin discloses (FIG. 1; Col 4, lines 39-51, display system 116 is a hardware system and comprises one or more display devices ... images 120 may be displayed on display system 116 through graphical user interface 122 in mobile inspection device 112; Col 5, lines 35-38, field of view 140 is what camera system 118 can generate images of, based on position 132 of mobile inspection device 112 with camera system 118. As shown in FIGS. 4-7, the graphical user interface displays the structure of the passenger cabin “Col 10, lines 64-67 to Col 11, lines 1-4, With reference now to FIG. 7, an illustration of the creation of a note in a graphical user interface is depicted in accordance with an illustrative embodiment. In this illustrative example, graphical user interface 700 is an example of an implementation for graphical user interface 122 in FIG. 1 and graphical user interface 336 in FIG. 3. In this example, graphical user interface 700 is displayed from a mobile inspection device, such as a pair of smart glasses”) wherein position information is input in accordance with a designation of a position on the image (Col 5, lines 49-55, the different notes in inspection system 102 are electronic notes rather than physical notes. For example, note 142 is an electronic note. Note 142 is assigned to location 152. In other words, location 152 may be located in or on vehicle 104 in this illustrative example. Location 152 is a location within or on vehicle 104, and is described using three-dimensional coordinates; Col 11, lines 5-11, additionally, human operator 704 has uttered a command, such as “create note”, in addition to the gesture of pointing to ceiling 708; Col 7, lines 16-21, FIG. 2 shows an illustration of a block diagram of a note is depicted in accordance with an illustrative embodiment. Note 200 is an example of one implementation for notes 146 in FIG. 1. As depicted note 200 includes graphical indicator 202, details 204, vehicle coordinates 206, and item identifier 208; Col 7, lines 22-67, graphical indicator 202 may include at least one of an icon, an image, a widget, text, an animation, color, or some other type of indicator that may be displayed in the graphical user interface to point to or show an association with an item; details 204 include at least one of text, voice, an image, a video, or some other content that may be used to explain the inconsistency; vehicle coordinates 206 identify the location for note 200 within or on vehicle 104 of FIG. 1. Vehicle coordinates 206 may be Cartesian coordinates, polar coordinates, or some other format based on a coordinate system for the vehicle).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the systems for creating a virtual reality scene taught by HEINEN incorporate the teachings of Laughlin, applying the vehicle inspection system taught by Laughlin to allow the user select a position on the image and provide the target position on the navigation interface. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify HEINEN according to the relied-upon teachings of Laughlin to obtain the invention as specified in claim.
Regarding claim 33, HEINEN discloses everything claimed as applied above (see claim 9).
However, HEINEN does not specifically disclose wherein position information indicates a position in the image.
In additional, Laughlin discloses (FIG. 1; Col 4, lines 39-51, display system 116 is a hardware system and comprises one or more display devices ... images 120 may be displayed on display system 116 through graphical user interface 122 in mobile inspection device 112; Col 5, lines 35-38, field of view 140 is what camera system 118 can generate images of, based on position 132 of mobile inspection device 112 with camera system 118. As shown in FIGS. 4-7, the graphical user interface displays the structure of the passenger cabin “Col 10, lines 64-67 to Col 11, lines 1-4, With reference now to FIG. 7, an illustration of the creation of a note in a graphical user interface is depicted in accordance with an illustrative embodiment. In this illustrative example, graphical user interface 700 is an example of an implementation for graphical user interface 122 in FIG. 1 and graphical user interface 336 in FIG. 3. In this example, graphical user interface 700 is displayed from a mobile inspection device, such as a pair of smart glasses”) wherein position information indicates a position in the image (Col 5, lines 49-55, the different notes in inspection system 102 are electronic notes rather than physical notes. For example, note 142 is an electronic note. Note 142 is assigned to location 152. In other words, location 152 may be located in or on vehicle 104 in this illustrative example. Location 152 is a location within or on vehicle 104, and is described using three-dimensional coordinates; Col 11, lines 5-11, additionally, human operator 704 has uttered a command, such as “create note”, in addition to the gesture of pointing to ceiling 708; Col 7, lines 16-21, FIG. 2 shows an illustration of a block diagram of a note is depicted in accordance with an illustrative embodiment. Note 200 is an example of one implementation for notes 146 in FIG. 1. As depicted note 200 includes graphical indicator 202, details 204, vehicle coordinates 206, and item identifier 208; Col 7, lines 22-67, graphical indicator 202 may include at least one of an icon, an image, a widget, text, an animation, color, or some other type of indicator that may be displayed in the graphical user interface to point to or show an association with an item; details 204 include at least one of text, voice, an image, a video, or some other content that may be used to explain the inconsistency; vehicle coordinates 206 identify the location for note 200 within or on vehicle 104 of FIG. 1. Vehicle coordinates 206 may be Cartesian coordinates, polar coordinates, or some other format based on a coordinate system for the vehicle).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the systems for creating a virtual reality scene taught by HEINEN incorporate the teachings of Laughlin, applying the vehicle inspection system taught by Laughlin to allow the user select a position on the image and provide the target position on the navigation interface. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify HEINEN according to the relied-upon teachings of Laughlin to obtain the invention as specified in claim.
Regarding claim 34, HEINEN discloses everything claimed as applied above (see claim 9).
However, HEINEN does not specifically disclose wherein position information indicates a position in a three-dimensional information in association with the image.
In additional, Laughlin discloses (FIG. 1; Col 4, lines 39-51, display system 116 is a hardware system and comprises one or more display devices ... images 120 may be displayed on display system 116 through graphical user interface 122 in mobile inspection device 112; Col 5, lines 35-38, field of view 140 is what camera system 118 can generate images of, based on position 132 of mobile inspection device 112 with camera system 118. As shown in FIGS. 4-7, the graphical user interface displays the structure of the passenger cabin “Col 10, lines 64-67 to Col 11, lines 1-4, With reference now to FIG. 7, an illustration of the creation of a note in a graphical user interface is depicted in accordance with an illustrative embodiment. In this illustrative example, graphical user interface 700 is an example of an implementation for graphical user interface 122 in FIG. 1 and graphical user interface 336 in FIG. 3. In this example, graphical user interface 700 is displayed from a mobile inspection device, such as a pair of smart glasses”) wherein position information indicates a position in a three-dimensional information in association with the image (Col 5, lines 49-55, the different notes in inspection system 102 are electronic notes rather than physical notes. For example, note 142 is an electronic note. Note 142 is assigned to location 152. In other words, location 152 may be located in or on vehicle 104 in this illustrative example. Location 152 is a location within or on vehicle 104, and is described using three-dimensional coordinates; Col 11, lines 5-11, additionally, human operator 704 has uttered a command, such as “create note”, in addition to the gesture of pointing to ceiling 708; Col 7, lines 16-21, FIG. 2 shows an illustration of a block diagram of a note is depicted in accordance with an illustrative embodiment. Note 200 is an example of one implementation for notes 146 in FIG. 1. As depicted note 200 includes graphical indicator 202, details 204, vehicle coordinates 206, and item identifier 208; Col 7, lines 22-67, graphical indicator 202 may include at least one of an icon, an image, a widget, text, an animation, color, or some other type of indicator that may be displayed in the graphical user interface to point to or show an association with an item; details 204 include at least one of text, voice, an image, a video, or some other content that may be used to explain the inconsistency; vehicle coordinates 206 identify the location for note 200 within or on vehicle 104 of FIG. 1. Vehicle coordinates 206 may be Cartesian coordinates, polar coordinates, or some other format based on a coordinate system for the vehicle).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the systems for creating a virtual reality scene taught by HEINEN incorporate the teachings of Laughlin, applying the vehicle inspection system taught by Laughlin to allow the user select a position on the image and provide the target position on the navigation interface. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify HEINEN according to the relied-upon teachings of Laughlin to obtain the invention as specified in claim.
Regarding claim 35, HEINEN discloses everything claimed as applied above (see claim 17).
However, HEINEN does not specifically disclose wherein position information indicates a position in the image.
In additional, Laughlin discloses (FIG. 1; Col 4, lines 39-51, display system 116 is a hardware system and comprises one or more display devices ... images 120 may be displayed on display system 116 through graphical user interface 122 in mobile inspection device 112; Col 5, lines 35-38, field of view 140 is what camera system 118 can generate images of, based on position 132 of mobile inspection device 112 with camera system 118. As shown in FIGS. 4-7, the graphical user interface displays the structure of the passenger cabin “Col 10, lines 64-67 to Col 11, lines 1-4, With reference now to FIG. 7, an illustration of the creation of a note in a graphical user interface is depicted in accordance with an illustrative embodiment. In this illustrative example, graphical user interface 700 is an example of an implementation for graphical user interface 122 in FIG. 1 and graphical user interface 336 in FIG. 3. In this example, graphical user interface 700 is displayed from a mobile inspection device, such as a pair of smart glasses”) wherein position information indicates a position in the image (Col 5, lines 49-55, the different notes in inspection system 102 are electronic notes rather than physical notes. For example, note 142 is an electronic note. Note 142 is assigned to location 152. In other words, location 152 may be located in or on vehicle 104 in this illustrative example. Location 152 is a location within or on vehicle 104, and is described using three-dimensional coordinates; Col 11, lines 5-11, additionally, human operator 704 has uttered a command, such as “create note”, in addition to the gesture of pointing to ceiling 708; Col 7, lines 16-21, FIG. 2 shows an illustration of a block diagram of a note is depicted in accordance with an illustrative embodiment. Note 200 is an example of one implementation for notes 146 in FIG. 1. As depicted note 200 includes graphical indicator 202, details 204, vehicle coordinates 206, and item identifier 208; Col 7, lines 22-67, graphical indicator 202 may include at least one of an icon, an image, a widget, text, an animation, color, or some other type of indicator that may be displayed in the graphical user interface to point to or show an association with an item; details 204 include at least one of text, voice, an image, a video, or some other content that may be used to explain the inconsistency; vehicle coordinates 206 identify the location for note 200 within or on vehicle 104 of FIG. 1. Vehicle coordinates 206 may be Cartesian coordinates, polar coordinates, or some other format based on a coordinate system for the vehicle).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the systems for creating a virtual reality scene taught by HEINEN incorporate the teachings of Laughlin, applying the vehicle inspection system taught by Laughlin to allow the user select a position on the image and provide the target position on the navigation interface. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify HEINEN according to the relied-upon teachings of Laughlin to obtain the invention as specified in claim.
Regarding claim 36, HEINEN discloses everything claimed as applied above (see claim 17).
However, HEINEN does not specifically disclose wherein position information indicates a position in a three-dimensional information in association with the image.
In additional, Laughlin discloses (FIG. 1; Col 4, lines 39-51, display system 116 is a hardware system and comprises one or more display devices ... images 120 may be displayed on display system 116 through graphical user interface 122 in mobile inspection device 112; Col 5, lines 35-38, field of view 140 is what camera system 118 can generate images of, based on position 132 of mobile inspection device 112 with camera system 118. As shown in FIGS. 4-7, the graphical user interface displays the structure of the passenger cabin “Col 10, lines 64-67 to Col 11, lines 1-4, With reference now to FIG. 7, an illustration of the creation of a note in a graphical user interface is depicted in accordance with an illustrative embodiment. In this illustrative example, graphical user interface 700 is an example of an implementation for graphical user interface 122 in FIG. 1 and graphical user interface 336 in FIG. 3. In this example, graphical user interface 700 is displayed from a mobile inspection device, such as a pair of smart glasses”) wherein position information indicates a position in a three-dimensional information in association with the image (Col 5, lines 49-55, the different notes in inspection system 102 are electronic notes rather than physical notes. For example, note 142 is an electronic note. Note 142 is assigned to location 152. In other words, location 152 may be located in or on vehicle 104 in this illustrative example. Location 152 is a location within or on vehicle 104, and is described using three-dimensional coordinates; Col 11, lines 5-11, additionally, human operator 704 has uttered a command, such as “create note”, in addition to the gesture of pointing to ceiling 708; Col 7, lines 16-21, FIG. 2 shows an illustration of a block diagram of a note is depicted in accordance with an illustrative embodiment. Note 200 is an example of one implementation for notes 146 in FIG. 1. As depicted note 200 includes graphical indicator 202, details 204, vehicle coordinates 206, and item identifier 208; Col 7, lines 22-67, graphical indicator 202 may include at least one of an icon, an image, a widget, text, an animation, color, or some other type of indicator that may be displayed in the graphical user interface to point to or show an association with an item; details 204 include at least one of text, voice, an image, a video, or some other content that may be used to explain the inconsistency; vehicle coordinates 206 identify the location for note 200 within or on vehicle 104 of FIG. 1. Vehicle coordinates 206 may be Cartesian coordinates, polar coordinates, or some other format based on a coordinate system for the vehicle).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the systems for creating a virtual reality scene taught by HEINEN incorporate the teachings of Laughlin, applying the vehicle inspection system taught by Laughlin to allow the user select a position on the image and provide the target position on the navigation interface. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify HEINEN according to the relied-upon teachings of Laughlin to obtain the invention as specified in claim.
Allowable Subject Matter
Claims 28-29 and 40 are allowed.
The following is an examiner’s statement of reasons for allowance:
Applicant provided paragraphs of the specification corresponding to the features recited in claims 28 and 40 in the Remarks. Accordingly, the 35 U.S.C. 112 rejections of claims 28-29 and 40 have been withdrawn.
Examiner has completed the additional search. However, the new search results and the prior art references cited in the previous Office Action failed to show the obviousness of the claims as a whole. None of the prior art cited alone or in combination provides the motivation to teach the limitations recited in claim 28. Accordingly, claim 28 is allowed.
Dependent claims 29 and 40 depend from independent claim 28 and are allowable for the same reasons as stated above.
Allowable Subject Matter
Claims 22-23 and 39 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Xilin Guo whose telephone number is (571)272-5786. The examiner can normally be reached Monday - Friday 9:00 AM-5:30 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XILIN GUO/ Primary Examiner, Art Unit 2616