DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 7-9, 12-13, 16, 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ishikawa (US 20230281897 A1), hereinafter Ishikawa, in view of Asaban (WO 2024176154 A2), hereinafter Asaban.
Regarding claim 1, Ishikawa teaches an information processing system comprising at least one processor, wherein the processor is configured to: acquire a first visual field image showing a visual field of a first user and a second visual field image showing a visual field of a second user (Fig. 4-5, paragraph 103, wherein display apparatus displaying a visual field image generated based on the visual field information of multiple users is interpreted as acquiring a first and a second visual field image showing a visual field of a first and second user); perform control of causing a second display device viewed by the second user to display first interest information of the first visual field image (Fig. 6, paragraph 108-114, wherein the visual field images of other users that share their content are displayed to the user, which is interpreted as having a first display device display a second visual field image from another user); and perform control of causing a first display device viewed by the first user to display second interest of the second visual field image (paragraph 108, wherein users being able to share their virtual field images to other users suggests that the first user can also share its visual field image to be displayed by a second user).
Ishikawa does not teach obtaining interest information indicating a position of at least one region of interest included in the visual field image.
Asaban teaches obtaining interest information indicating a position of at least one region of interest included in the visual field image (Fig. 3, paragraph 127, wherein reading, processing, and cropping image data corresponding to a user’s FOV to display the portion of data corresponding to the region of interest is interpreted as displaying interest information indicating a position of a region of interest, and the image data corresponding to an FOV is interpreted as a visual field image).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ishikawa with the teaching of Asaban for this system for displaying regions of interests for visual field images. Ishikawa discusses a system where users can see the visual fields of other users, making it easier for users to communicate and share user’s visual field information while viewing the same content. Asaban discusses identifying regions of interest from a user’s visual field, in order to better identify and display data related to performing medical procedures. Both references discuss using the visual field data of a user wearing a head-mounted display. While Ishikawa does not explicitly discuss using its system for medical imaging, it would be obvious that the techniques of identifying a user’s line of sight to determine visual field data could be used for any type of HMD imaging, including for medical imaging. Furthermore, while Asaban does not discuss having multiple users capturing separate visual field images, it does discuss outputting the visual field images to an external, secondary display. It would be obvious to have multiple users operate the HMD described in Asaban, and to have their images outputted to separate displays. As both references discuss identifying a user’s visual field image from an HMD, and discuss non limiting art on displaying additional data to the user of the HMD, it would be obvious to combine them.
Regarding claim 7, Ishikawa in view of Asaban discloses the system of claim 1. Additionally, Asaban teaches the information processing system according to claim 1, wherein the processor is configured to: perform control of displaying, as the first interest information, an image on a left side among images obtained by dividing the first visual field image into two parts of left and right on a left side of a region in which the visual field of the second user is displayed on the second display device (Fig. 2, paragraph 127-128, wherein image data is obtained with respect to a given FOV, which is interpreted as a visual field image, and wherein the visual field image is displayed on a left and right display 30, wherein the left and right displays may show a stereoscopic view of the visual field image, which is interpreted as the visual image being divided into a left and right image to be displayed on a left and right side of a display device); and perform control of displaying, as the first interest information, an image on a right side among the images obtained by dividing the first visual field image into two parts of left and right on a right side of the region in which the visual field of the second user is displayed on the second display device (Fig. 2, paragraph 127-128, wherein the visual field image is displayed on a left and right display 30, wherein the left and right displays may show a stereoscopic view of the visual field image which is interpreted as the visual image being divided into a left and right image to be displayed on a left and right side of a display device).
The motivation to combine would be the same as that set forth in claim 1.
Regarding claim 8, Ishikawa in view of Asaban discloses the system of claim 1. Additionally, Ishikawa teaches the information processing system according to claim 1, wherein the processor is configured to: acquire the first visual field image over time; determine whether or not there is a change in a position or an orientation of the first region of interest from the first visual field image acquired over time (Fig. 22, paragraph 241, detecting visual field image line of sight movement over time, which is interpreted as acquiring a visual field image over time to determine if there is a change in position or orientation, and suggests including detecting if a region of interest within the visual field image moves over time); and perform control of causing, in a case where the determination is affirmative, the second display device to display information indicating that there is the change in the position or the orientation of the first region of interest (Fig. 22, paragraph 241-242, wherein determining if the visual field image has moved over time and moving the visual field image according to that movement is interpreted as displaying information indicating that there is change in position or orientation of the visual field image, including any regions of interest within the visual field image).
Regarding claim 9, Ishikawa in view of Asaban discloses the system of claim 1. Additionally, Ishikawa teaches the information processing system according to claim 1, wherein the processor is configured to: acquire first visual line information of the first user (paragraph 93, wherein the visual field image is determined from a user’s line-of-sight, which is interpreted as the visual field image containing acquired visual line information); and perform control of causing the second display device to display the first visual line information of the first user (Fig. 6, paragraph 108-114, wherein the visual field images of other users are displayed to the user, which is interpreted as having a second display device display a first visual field from a given user).
Regarding claim 12, Ishikawa in view of Asaban discloses the system of claim 1. Additionally, Ishikawa teaches the information processing system according to claim 1, wherein the processor is configured to: acquire a voice of the first user (paragraph 96, wherein speech is obtained via voice input from a user); convert the voice of the first user into text information; and perform control of causing the second display device to display the text information (Fig. 13, paragraph 152-153, wherein text is displayed corresponding to another user’s speech, which is displayed to a user, which is interpreted as converting a voice of a user into text information and displaying that text to another user).
Regarding claim 13, Ishikawa in view of Asaban discloses the system of claim 1. Additionally, Ishikawa discloses the information processing system according to claim 1, wherein the processor is configured to: generate a second composite image obtained by combining the second visual field image with the information from the first visual field image; generate a first composite image obtained by combining the first visual field image with the information from the first visual field image (Fig. 5, paragraph 108-109, wherein the wide area image displaying the visual fields of a first and second user is interpreted as a composite image formed by combining the second visual field image with a first visual field image); perform control of causing the second display device to display the second composite image; and perform control of causing the first display device to display the first composite image (paragraph 107-108, wherein the wide area image is displayed depicting the visual field images of other users to a given user, and suggests a first and second user can view a separate first and second composite image).
Ishikawa does not teach obtaining information indicating the position of a region of interest from a visual field image.
Asaban teaches obtaining information indicating the position of a region of interest from a visual field image (Fig. 3, paragraph 127, wherein reading, processing, and cropping image data corresponding to a user’s FOV to display the portion of data corresponding to the region of interest is interpreted as obtaining interest information indicating a position of a region of interest, and the image data corresponding to an FOV is interpreted as a visual field image).
The motivation to combine is the same as that set forth for claim 1.
Regarding claim 16, Ishikawa in view of Asaban discloses the system of claim 1. Additionally, Ishikawa teaches the information processing system according to claim 1, wherein the processor is configured to perform control of causing a third display device viewed by a third user to display the first interest information (Fig. 6, paragraph 108-114, wherein the visual field images of multiple other users sharing visual field information are displayed to the user, which suggests that a third user can display visual field information obtained from a first user).
Regarding claim 19, Ishikawa teaches an information processing method executed by a computer, the information processing method comprising: acquiring a first visual field image showing a visual field of a first user and a second visual field image showing a visual field of a second user (Fig. 4-5, paragraph 103, wherein display apparatus displaying a visual field image generated based on the visual field information of multiple users is interpreted as acquiring a first and a second visual field image showing a visual field of a first and second user); performing control of causing a second display device viewed by the second user to display first interest information of the first visual field image (Fig. 6, paragraph 108-114, wherein the visual field images of other users that share their content are displayed to the user, which is interpreted as having a first display device display a second visual field image from another user); and performing control of causing a first display device viewed by the first user to display second interest of the second visual field image (paragraph 108, wherein users being able to share their virtual field images to other users suggests that the first user can also share its visual field image to be displayed by a second user).
Ishikawa does not teach obtaining interest information indicating a position of at least one region of interest included in the visual field image.
Asaban teaches obtaining interest information indicating a position of at least one region of interest included in the visual field image (Fig. 3, paragraph 127, wherein reading, processing, and cropping image data corresponding to a user’s FOV to display the portion of data corresponding to the region of interest is interpreted as displaying interest information indicating a position of a region of interest, and the image data corresponding to an FOV is interpreted as a visual field image).
The motivation to combine would be the same as that set forth for claim 1.
Regarding claim 20, Ishikawa teaches a non-transitory computer-readable storage medium storing an information processing program (Fig. 26, paragraph 289) causing a computer to execute a process comprising: acquiring a first visual field image showing a visual field of a first user and a second visual field image showing a visual field of a second user (Fig. 4-5, paragraph 103, wherein display apparatus displaying a visual field image generated based on the visual field information of multiple users is interpreted as acquiring a first and a second visual field image showing a visual field of a first and second user); performing control of causing a second display device viewed by the second user to display first interest information of the first visual field image (Fig. 6, paragraph 108-114, wherein the visual field images of other users that share their content are displayed to the user, which is interpreted as having a first display device display a second visual field image from another user); and performing control of causing a first display device viewed by the first user to display second interest of the second visual field image (paragraph 108, wherein users being able to share their virtual field images to other users suggests that the first user can also share its visual field image to be displayed by a second user).
Ishikawa does not teach obtaining interest information indicating a position of at least one region of interest included in the visual field image.
Asaban teaches obtaining interest information indicating a position of at least one region of interest included in the visual field image (Fig. 3, paragraph 127, wherein reading, processing, and cropping image data corresponding to a user’s FOV to display the portion of data corresponding to the region of interest is interpreted as displaying interest information indicating a position of a region of interest, and the image data corresponding to an FOV is interpreted as a visual field image).
The motivation to combine would be the same as that set forth for claim 1.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Ishikawa in view of Asaban as applied to claim 1 above, and further in view of Palushi (US 12594125 B2), hereinafter Palushi.
Regarding claim 2, Ishikawa in view of Asaban discloses the system of claim 1. Additionally, Palushi teaches the information processing system according to claim 1, wherein the processor is configured to perform control of causing the second display device to display, as the first interest information, an image in which the first visual field image is made to be semi-transparent (Col. 13, lines 4-21, wherein overlaying markers within a user’s field of view displayed on a transparent display is interpreted as the displayed marker image being made to be semi-transparent, wherein the overlaid markers are rendered overlaid on objects based on the user’s field of view, which is interpreted as the overlay markers being a visual field image).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ishikawa in view of Asaban with the teachings of Palushi for this system for displaying regions of interests for visual field images. Ishikawa discusses a system where users can see the visual fields of other users, making it easier for users to communicate and share user’s visual field information while viewing the same content. Asaban discusses identifying regions of interest from a user’s visual field, in order to better identify and display data related to performing medical procedures. Similarly, Palushi discusses identifying regions of interest from a user’s field of vision, in order to overlay markers and useful information to a user performing a task such as medical procedures. Both Asaban and Palushi discuss using a head-mounted display to identify regions of interest, and display useful information for the purposes of medical procedures. Additionally, all three references discuss using the visual field data of a user wearing a head-mounted display. While Ishikawa does not explicitly discuss using its system for medical imaging, it would be obvious that the techniques of identifying a user’s line of sight to determine visual field data could be used for any type of HMD imaging, including for medical imaging. Furthermore, while Asaban and Palushi does not discuss having multiple users capturing separate visual field images, they do both discuss outputting the visual field images to an external, secondary display. It would be obvious to have multiple users operate the HMD described in Asaban and Palushi, and to have their images outputted to separate displays. As Asaban and Palushi discuss analogous art, and all three references discuss identifying a user’s visual field image from an HMD, and discuss non limiting art on displaying additional data to the user of the HMD, it would be obvious to combine them.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Ishikawa in view of Asaban as applied to claim 1 above, and further in view of Vetter (US 20220343586 A1), hereinafter Vetter.
Regarding claim 3, Ishikawa in view of Asaban discloses the system of claim 1. Additionally, Asaban teaches extracting the at least one first region of interest from the first visual field image (Fig. 3, paragraph 127, wherein reading, processing, and cropping image data corresponding to a user’s FOV to display the portion of data corresponding to the region of interest is interpreted as extracting a region of interest from a visual field image).
Neither Ishikawa nor Asaban teaches causing the second display device to display, as the first interest information, information indicating a contour of the extracted at least one first region of interest.
Vetter teaches displaying information indicating a contour of the extracted at least one first region of interest (Figs 11-13, paragraph 141, wherein contour lines of a given region of interest is displayed, which is interpreted as displaying information indicating a contour of the extracted region of interest).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ishikawa in view of Asaban with the teachings of Vetter for this system for displaying regions of interests for visual field images. Ishikawa discusses a system where users can see the visual fields of other users, making it easier for users to communicate and share user’s visual field information while viewing the same content. Asaban discusses identifying regions of interest from a user’s visual field, in order to better identify and display data related to performing medical procedures. Additionally, Vetter discusses a method for better estimating distances in regions of interest, to better increase the accuracy of rendered images. Both Ishikawa and Asaban discuss using the visual field data of a user wearing a head-mounted display. Meanwhile, Vetter discusses using obtained images from VR or AR devices such as a head-mounted display, and better visualizing the distances within that image to improve rendering accuracy. While Ishikawa does not explicitly discuss using its system for medical imaging, it would be obvious that the techniques of identifying a user’s line of sight to determine visual field data could be used for any type of HMD imaging, including for medical imaging. Furthermore, while Asaban does not discuss having multiple users capturing separate visual field images, they do both discuss outputting the visual field images to an external, secondary display. Both references would appreciate the distance estimation taught in Vetter, in order to render more accurate images and obtain more detailed information about the visual field and regions of interest. As both Ishikawa and Asaban discuss identifying a user’s visual field image from an HMD, and discuss non limiting art on displaying additional data to the user of the HMD, and Vetter discusses a nonlimiting way to improve the accuracy of images obtained from an HMD, it would be obvious to combine them.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JORDAN W YICK whose telephone number is (571)272-4063. The examiner can normally be reached M-F 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at (571) 272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JORDAN WAN YICK/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights