DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see pg. 7, filed 11/26/25, with respect to the status of the claims is hereby acknowledged.
Applicant’s arguments, see pg. pg. 7-8, filed 11/26/25, with respect to the rejection of claims 1, 2, 4-10, and 12-21 under 35 U.S.C. 103 have been fully considered. In particular, the applicant traverses the rejections and argues the following:
The cited references fail to disclose or suggest all of the features of the claims, and provide no apparent reason for modification to include such features….In rejecting claim 1, the Office acknowledged that "[r]egarding wherein the information about the ROls indicates a region of overlap between two or more of the ROls, wherein the wherein the transmission circuitry sends the image data of the respective ROls through a mutually common virtual channel - Sato, Fukuhara and Nxp Semiconductors do not disclose an overlapping feature as claimed."…To support the rejection of claim 1, the Office asserted that "[i]n an analogous art, Vembar teaches prioritizing the transmission of regions of interest (ROls) utilizing parallel processing wherein para 55-57 - utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams; see also para 166 dynamically resize and assign priorities to viewports and transmit rendered viewports as they are ready (e.g. while other viewports render)."…
Applicant respectfully disagrees. Even with the addition of Vembar, the Office still has not provided any evidence in the cited references that teaches "wherein the transmission circuitry sends the image data of the respective ROIs through a mutually common virtual channel," as recited by claim 1. The Office's cited paragraphs [0055]-[0057] of Vembar appear to teach the utilization of multiple virtual channels, which is different from "a mutually common virtual channel."3 Thus, the Office failed to establish a primafacie case of obviousness with respect to claim 1 because the cited references (including the new addition of Vembar) fail to disclose or suggest all of the features of claim 1, and provide no apparent reason for modification to include such features.
The examiner respectfully disagrees and incorporates by reference the findings of fact made in the PTAB Decision herein.
The Examiner’s findings of fact is based on the interpretation of the claim terms (e.g., wherein the transmission circuitry sends the image data of the respective ROIs through a mutually common virtual channel) because it is a key aspect in the analysis of the principles and elements taught by the prior art and will support the inferences that a person of ordinary skill would have made from the combination of teachings of the prior art of record. Based on the broadest reasonable interpretation of the claim limitations, the limitation would be interpreted as the image data of the respective ROIs through a mutually common virtual channel all share the same virtual channel. Alternatively, even if the applicant clarifies on the record that the examiner’s interpretation of the claim is incorrect, an alternate interpretation of the claim would also be obvious based on the combined teachings of the prior art. Stated differently, even if multiple virtual channels are utilized to transmit each of the respective ROIs, then that interpretation of the claim would also be rendered obvious in view of the combined teachings.
For example, in response to the applicant’ arguments, the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). More importantly, on the issue of obviousness, the Supreme Court stated that when a patent simply arranges old elements with each performing the same function it had been known to perform and yields no more than one would expect from such an arrangement, the combination is obvious. KSR International Co. v. Teleflex Inc., 550 U.S. 398, 417, 82 USPQ2d 1385 (2007) (citing Sakraida v. AG Pro, Inc., 425 U.S. 273, 96 S. Ct. 1532, 47 L. Ed. 2d 784 (1976)). The analysis of a rejection on obviousness grounds need not seek out precise teachings directed to the specific subject matter of the challenged claim, for a court can take account of the inferences and creative steps that a person of ordinary skill in the art would employ. See id. at 418. The obvious analysis cannot be confined by a formalistic conception of the words teaching, suggestion, and motivation. Id. at 419. Further, the Court stated that common sense teaches, however, that familiar items may have obvious uses beyond their primary purposes, and in many cases a person of ordinary skill will be able to fit the teachings of multiple patents together like pieces of a puzzle. See id. at 420.
The Court further reiterated that in circumstances where the combination of two pre-existing elements did no more than they would in separate, sequential operation, the patent failed under 35 U.S.C. § 103. See id. at 416-417 (citing Anderson's-Black Rock, Inc. v. Pavement Salvage Co., 396 U.S. 57, 90 S. Ct. 305, 24 L. Ed. 2d 258 (1969)).
In view of the principles of law discussed above, the Examiner asserts that applicant’s limitation (e.g., wherein the transmission circuitry sends the image data of the respective ROIs through a mutually common virtual channel) would also be rendered obvious. For example, applicant’s claim limitation, in either interpretation discussed above, would be rendered obvious even if transmission circuitry sending the image data of the respective ROIs through using multiple virtual channels of using one virtual channel that is common because a transmission circuitry that separates data into multiple virtual channels or uses one virtual channel. This is a corollary to a circumstance where the combination of two pre-existing elements did no more than they would in a separate operations as discussed in KSR. The Examiner asserts that merely combining multiple pieces of data to be transmitted on a common virtual channel or separating multiple pieces of data to be transmitted using multiple common virtual channel would do no more than they would in separate operation. By the same token, splitting or separating data into multiple virtual channels based on pre-existing or known parameters would have also been obvious. Therefore, Examiner does not agree with the applicant’s argument that the rejection of claim 1 under 35 U.S.C. § 103(a) is improper.
Therefore, the interpretation of a the limitations (e.g., wherein the transmission circuitry sends the image data of the representative ROIs through a mutually common virtual channel) does not take into consideration elements/limitations of the “examples” disclosed in Appellant’s originally filed specification that are not explicitly recited in the claims. MPEP § 2111.01 states that “(o)ne must bear in mind that, especially in nonchemical cases, the words in a claim are generally not limited in their meaning by what is shown or disclosed in the specification. See, e.g., Liebel-Flarsheim Co. v. Medrad Inc., 358 F.3d 898, 906, 69 USPQ2d 1801, 1807 (Fed. Cir. 2004)(discussing recent cases wherein the court expressly rejected the contention that if a patent describes only a single embodiment, the claims of the patent must be construed as being limited to that embodiment).” As such, the claim has been interpreted in view of the specification without unnecessarily importing limitations from the specification into the claims.
In the teachings of Hwang, the prior art teaches enhancement layers for transmitting data on a plurality of times. The examiner further previously used the teachings of Thomas to further evidence how a person of ordinary skill in the art would have understood an enhancement layer as taught by Hwang in relation to overlapping of ROIs discussed in Thomas. For example, Hwang does teach that the panoramic image may be divided into left, center and right regions. As shown in the figure, the panoramic image may be divided into a plurality of images in a mosaic form (Hwang para 115) and further teaches indicating the number of Regions of Interest being transmitted (para 180-183, 222). More importantly, Hwang discloses elements with respect to enhancement signals of the ROI is merged with a base layer (para 83, 92-96, 111-112, 162-166). As such, the enhanced region of interest is composed of sections of different regions of the display image that are of interest to the user. For example, the enhanced region of interest is a portion of each of the tiles that are of interest to the receiving user. The examiner will rely on the teachings of Nxp Semiconductors recognizes that the purpose of a virtual channel identifier is to provide separate channels for different data flows that are interleaved in the data stream (Section 2.2.2.3) and Nxp Semiconductors Section 2.2.2 support for up to four interleaved virtual channels).
More importantly, Vembar teaches prioritizing the transmission of regions of interest (ROIs) utilizing parallel processing wherein para 55-57 - utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams; see also para 166 dynamically resize and assign priorities to viewports and transmit rendered viewports as they are ready (e.g. while other viewports render). For example, the PPU 713 may include similar components and/or features as the parallel processor 200, further configured with a graphics viewport apparatus as described herein. The system 700 may also be adapted to work with a stereo head mounted system such as, for example, the system described in connection with FIGS. 11-15 below. In particular, the HMII 1100 described in more detail below may include a gaze tracker to provide gaze/focus information to the frame divider 721 and/or viewport prioritizer 722. See also para 170 a prioritized region of interest (ROI) for wireless VR use cases and applications, so there is a priority associated with which region of the image is being encoded and decoded. In some embodiments, eye tracking information may be utilized to identify a prioritized ROI. Some embodiments may provide similar benefits for a wired VR system (e.g. where transmission occurs over a cable). See also para 180 some embodiments may provide individual rendering frequencies for a wireless VR. Objects in the periphery may not need to be rendered at the same rate as the center of the screen. Some embodiments may render different blocks of an image at different fps for prioritized rendering and display. Advantageously, some embodiments may reduce the amount of work being done or allow doing more work at the center or ROI of the image. Some embodiments may blend content at the edges of the different viewports (e.g. blending new data with stale pixel data). For example, a HMD may include sensors to provide eye/gaze tracking information which the GPU/CPU may use to change the ROI dynamically and adjust the viewports.
See prior art made of record but not relied upon to avoid duplicative references: Thomas et al., PG Pub 2018/0295400 (hereafter Thomas) teaches identifying multiple regions of interest with overlapping content (para 44-48) and including the data in payload content (para 59, 117, 121, 134).
For at least these reasons, claim 1 is not patentable under 35 U.S.C. § 103. Independent claims 9 and 17 recite features that are similar to the features of claim 1. Therefore, claims 9 and 17 are respectively not distinct from the cited references for at least the reasons provided above regarding claim 1, wherein each claim is considered according to its particular recitations. The dependent claims, i.e., claims 2, 4-8, 10, 12-16, and 18-21, are respectively also not distinct from the cited references by their incorporation of the features of their respective base claims, as well as for their separately recited patentably distinct features. For at least the reasons above, claims 1, 2, 4-10, and 12-21 are not patentable under 35 U.S.C. § 103. Accordingly, the rejections are maintained.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
With respect to newly added claim 22 (i.e., wherein the ROIs include a first ROI and one or more additional ROIs, and the image data includes a first portion of the image data corresponding the first ROI, and a second portion of the image data corresponding to less than all of the one or more additional ROIs, wherein the information about the ROIs indicates a region of overlap between the first ROI and the one or more additional ROIs, wherein the first portion of the image data includes image data of the region of overlap, wherein the second portion of the image data does not include the image data of the region of overlap, and wherein the first portion and the second portion of the image data are each less than an entirety of the image), the claim is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph. See MPEP §2163.04 , e.g., Hyatt v. Dudas, 492 F.3d 1365, 1370, n.4 (Fed. Cir. 2007) (citing MPEP § 2163.04 which provides that a “simple statement such as ‘applicant has not pointed out where the new (or amended) claim is supported, nor does there appear to be a written description of the claim limitation ‘___’ in the application as filed’ may be sufficient where the claim is a new or amended claim, the support for the limitation is not apparent, and applicant has not pointed out where the limitation is supported.”); see also MPEP §§ 714.02 and 2163.06 (“Applicant should ... specifically point out the support for any amendments made to the disclosure.”); and MPEP § 2163.04 states “If applicant amends the claims and points out where and/or how the originally filed disclosure supports the amendment(s), and the examiner finds that the disclosure does not reasonably convey that the inventor had possession of the subject matter of the amendment at the time of the filing of the application, the examiner has the initial burden of presenting evidence or reasoning to explain why persons skilled in the art would not recognize in the disclosure a description of the invention defined by the claims.”). The applicant has not pointed out where the new (or amended) claim is supported, nor does there appear to be a written description of said representative claim limitation in the application as filed.
In particular, the newly added claim recites, inter alia, “wherein the first portion of the image data includes image data of the region of overlap, wherein the second portion of the image data does not include the image data of the region of overlap, and wherein the first portion and the second portion of the image data are each less than an entirety of the image.” Therefore, the new claim provides substantive changes which need to be properly analyzed in view of the originally filed specification but the applicant has not clearly pointed out where and/or how the originally filed disclosure supports the amendment(s). For example, in applicant’s originally filed specification, paragraph [0030] mentions the omission of regions based on priority data, however, the examiner is unable to determine whether the disclosure is related to claim 22 and whether the priority identifier is essential to the invention and whether the elements should be analyzed under MPEP §2172.01. The broadest reasonable interpretation of claim elements are discussed in the rejection of claim 22 infra. Correction is required.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim 1-2, 4-10, 12-22 are rejected under 35 U.S.C. 103 as being unpatentable over Sato; Kazushi US 20150023407 A1 (hereafter Sato) and in further view of Fukuhara; Takahiro et al. US 20130163889 A1 (hereafter Fukurama) and in further view of Raveendran; Vijayalakshmi R. et al. US 20110032334 A1 (hereafter Raveendran) and in further view of Nxp Semiconductors: "MIPI-CSI2 Peripheral on i.MX6 MPUs", 1 July 2016 (2016-07-01), XP055314398 (hereafter Nxp Semiconductors) and in further view of HWANG; Soojin et al. US 20160337706 A1 (hereafter Hwang) and in further view of Vembar; Deepak S. et al. US 20180288423 A1 (hereafter Vembar).
Regarding claim 1, “a picture transmission device comprising: a transmission circuitry that sends image data of ROIs (Regions Of Interest) in an image in Payload Data of a Long Packet and sends information about the ROIs in Embedded Data, ” Sato para 155-165 teaches (sends image data of tiles of interest [i.e. regions]. Whereas Sato does not specifically reference a Long Packet with respect to Payload Data, Sato does teach transmission of tiles of interest using encoding schemes such as MPEG-2. See also Sato Fig. 9 disclosing packetizing image data comprising header and payload data. With respect to circuitry, Sato para 252 references units understood as processing units of an encoding process which typically comprises hardware, software, circuitry or a combination of hardware, software, circuitry.
In an analogous art, Fukuhara teaches the deficiency of Sato (Fig. 2, 10A-10C and para 155-162 wherein the Body Partition comprises payload data for encoded image blocks).
Whereas Sato discloses a schematic configuration of a mobile telephone applying the claimed invention and Fukuhara teaches an image processing apparatus, the motivation to modify the teachings of Sato and Fukuhara are further evidenced by Raveendran disclosing an invention for a mobile image device preparing video data in accordance with a display protocol utilizing packets comprising processed using mobile devices (see Raveendran Fig. 7-9 and para 165-175). Whereas Raveendran does not specifically user the term “Long Packet” as claimed, the prior art recognizes the use of Long Packets when utilizing a mobile device to transmit packets data – see Nxp Semiconductors (Section 2.2.2 2.2.2.1 and teaches utilizing Payload Data of a Long Packet). Nxp Semiconductors recognizes that the purpose of a virtual channel identifier is to provide separate channels for different data flows that are interleaved in the data stream (Section 2.2.2.3) and Nxp Semiconductors Section 2.2.2 support for up to four interleaved virtual channels).
Regarding wherein the information about the ROIs indicates a region of overlap between two or more of the ROIs, wherein the wherein the transmission circuitry sends the image data of the respective ROIs through a mutually common virtual channel - Sato, Fukuhara and Nxp Semiconductors do not disclose an overlapping feature as claimed. Hwang further teaches “ROIs indicates a region of overlap between two or more of the ROIs” and transmitting a plurality of pieces of third image data in Payload Data of a Long Packet and sends information about the respective ROIs in the image in Embedded Data, the plurality of pieces of third image data being obtained by omitting second image data of the region of overlap from a plurality of pieces of first image data of the ROIs in the image to avoid the second image data from being included redundantly in the plurality of pieces of first image data (see Hwang Fig. 16-17 and para 161-185 ROI Region overlaps the middle four regions of interest and the ROI info metadata for Residual data for ROI start point (X,Y) & (ROI_width X ROI height) and similarly in Fig. 17 and para 200-215 wherein ROI transmitted to the monitor comprises ROI which overlaps regions 3,4,5,6 renders obvious to discard/omit image data of the region of overlap from pieces of first image data of the ROIs in the image to avoid the redundant transmission of other regions in the plurality of pieces of first image data). With respect to an image divided into multiple regions of interest, Hwang teaches the divided images are transmitted through respective streams (para 12, 83, 131, 132, 175, 178, 217, 225, 310, 381) and also teaches that a plurality of virtual channels are transmitted as part of a transport stream (see para 301, 315-317, 328-343. See also Hwang para 381 - each elementary stream with respect to each image included in the service may include stream_info_details( ), and the stream_info_details( ) may signal information about the streams with respect to the panoramic image. In the panorama service according to an embodiment of the present invention, information about each image may be signaled using the stream_info_details( ) according to an embodiment of the present invention even when the stream_type is MPEG2, AVC, HEVC or the like and Hwang para 386 - component_count field may indicate the number of components (streams) included in a relevant channel (service). Whereas Hwang as discussed supra discloses that multiple streams are multiplexed as a number of components (streams) included in a relevant channel (service), as discussed above, Nxp Semiconductors recognizes that the purpose of a virtual channel identifier is to provide separate channels for different data flows that are interleaved in the data stream (Section 2.2.2.3).
In an analogous art, Vembar teaches prioritizing the transmission of regions of interest (ROIs) utilizing parallel processing wherein para 55-57 - utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams; see also para 166 dynamically resize and assign priorities to viewports and transmit rendered viewports as they are ready (e.g. while other viewports render). For example, the PPU 713 may include similar components and/or features as the parallel processor 200, further configured with a graphics viewport apparatus as described herein. The system 700 may also be adapted to work with a stereo head mounted system such as, for example, the system described in connection with FIGS. 11-15 below. In particular, the HMII 1100 described in more detail below may include a gaze tracker to provide gaze/focus information to the frame divider 721 and/or viewport prioritizer 722. See also para 170 a prioritized region of interest (ROI) for wireless VR use cases and applications, so there is a priority associated with which region of the image is being encoded and decoded. In some embodiments, eye tracking information may be utilized to identify a prioritized ROI. Some embodiments may provide similar benefits for a wired VR system (e.g. where transmission occurs over a cable). See also para 180 some embodiments may provide individual rendering frequencies for a wireless VR. Objects in the periphery may not need to be rendered at the same rate as the center of the screen. Some embodiments may render different blocks of an image at different fps for prioritized rendering and display. Advantageously, some embodiments may reduce the amount of work being done or allow doing more work at the center or ROI of the image. Some embodiments may blend content at the edges of the different viewports (e.g. blending new data with stale pixel data). For example, a HMD may include sensors to provide eye/gaze tracking information which the GPU/CPU may use to change the ROI dynamically and adjust the viewports.
See prior art made of record but not relied upon to avoid duplicative references: Thomas et al., PG Pub 2018/0295400 (hereafter Thomas) teaches identifying multiple regions of interest with overlapping content (para 44-48) and including the data in payload content (para 59, 117, 121, 134).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato for a transmitting device with circuit components to send image data of tiles comprising regions of interest utilizing an encoding scheme, such as MPEG-2 for packetizing data by further incorporating known elements of Fukuhara’s invention for transmitting image data, comprising image blocks for regions in an image, using an encoding scheme which transmits a payload comprising the region information because the combination of known elements for transmitting encoding data using known encoding schemes would result in predictable results with a high likelihood of success and because a person of ordinary skill in the art would reasonably infer that payload data is typically utilized for transmitting graphics and or the actual intended message such as image frame content. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato and Fukuhara by further incorporating known elements of Raveendran for a mobile image device protocol for transmitting packets comprising regions of interest data because the prior art recognizes a mobile image device protocol for transmitting packet data comprising a packet structure utilizing Long Packets in order to utilize a standardized protocol to efficiently image content as taught by Nxp Semiconductors. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato, Fukuhara, Raveendran and Nxp Semiconductors by further incorporating known elements of Hwang for utilizing more than one channel to transmit regions of interest that would require the transmission of higher definition data as suggested by Hwang when transmitting extracted overlapping regions of interest and provided in a higher definition format than the image with the plurality ROI because Vembar teaches prioritizing the transmission of regions of interest (ROIs) utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams related to the same content comprising a frame divided into viewports.
Regarding claim 2, “wherein the transmission circuitry sends the image data of the respective ROIs through virtual channels different from each other” Sato, Fukuhara, and Raveendran are silent with respect to virtual channels but Nxp Semiconductors recognizes that the purpose of a virtual channel identifier is to provide separate channels for different data flows that are interleaved in the data stream (Section 2.2.2.3) and Nxp Semiconductors Section 2.2.2 support for up to four interleaved virtual channels). Vembar teaches prioritizing the transmission of regions of interest (ROIs) utilizing parallel processing wherein para 55-57 - utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams; see also para 166 dynamically resize and assign priorities to viewports and transmit rendered viewports as they are ready (e.g. while other viewports render). The motivation of combine Sato, Fukuhara, Raveendran, and Nxp Semiconductors is further evidenced by Hwang (para 380-399 regions of interest transmitted utilizing different streams pertaining to ROIs for an image.).
Regarding claim 4, “wherein the transmission circuitry puts data types of the respective ROIs in a packet header of the Payload Data, and sends the data types of the respective ROIs” is further rejected on obviousness grounds as discussed in the rejection of claim 1-2 wherein Raveendran Fig. 7-9 and para 165-175. Whereas Raveendran does not specifically user the term “Long Packet” as claimed, the prior art recognizes the use of Long Packets when utilizing a mobile device to transmit packets data – see Nxp Semiconductors (Section 2.2.2 2.2.2.1 and teaches utilizing Payload Data of a Long Packet and header). Sato teaches header indicates whether stream comprises a ROI parameter data indicates a type (wherein the broadest reasonable interpretation of a type comprises the definition of “something distinguishable as a variety”) (See also Sato para 117-118, 123, 155, 162, 182, 188, 248, 253).
Regarding claim 5, “wherein the transmission circuitry puts at least one of number of the ROIs included in the image, a region number of each of the ROIs, a data length of each of the ROIs, or an image format of each of the ROIs in the Payload Data, and sends the at least one of the number of the ROIs included in the image, the region number of each of the ROIs, the data length of each of the ROIs, or the image format of each of the ROIs” is further rejected on obviousness grounds as discussed in the rejection of claim 1-2 and 4 wherein Hwang teaches a region number in para 167.
Regarding claim 6, “wherein the transmission circuitry puts at least one of number of the ROIs included in the image, a region number of each of the ROIs, a data length of each of the ROIs, or an image format of each of the ROIs in a Short Packet, and sends the at least one of the number of the ROIs included in the image, the region number of each of the ROIs, the data length of each of the ROIs, or the image format of each of the ROIs” is further rejected on obviousness grounds as discussed in the rejection of claim 1-2, 4-5 wherein Hwang teaches a region number in para 167.
Regarding claim 7, “wherein the transmission circuitry sends the image data of the ROIs in an image data frame and sends the information about the ROIs in a header or a footer of the image data frame” is further rejected on obviousness grounds as discussed in the rejection of claim 1-2, 4-6 wherein Sato teaches image encoding device comprises a tile partitioning section for generating frames into memory (para 79, 90-93, 113, 122, 127-135, 146, 153, 156-160, 163, 169, 184-191 encoded stream of based layer and image of ROI tile is stored in the frame memory). See also Nxp Semiconductors Section 2.2.2 to 2.2.2.1 utilizing Payload Data of a Long Packet comprising a header and footer.
Regarding claim 8, “wherein the transmission circuitry sends a signal in MIPI (Mobile Industry Processor Interface) CSI (Camera Serial Interface)-2 specification, MIPI CSI-3 specification, or MIPI DSI (Display Serial Interface) specification” Sato and Fukuhara are silent with respect to MIP specifications. However, in an analogous art, Raveendran teaches a transmitting ROI utilizing a MIPI transmitter (Fig. 7-9 and para 53, 165-175). Even where Raveendran does not specifically CSI-2 or CSI-3, a person of ordinary skill would reasonably infer that a MIPI transceiver utilizing any amended MIPI standard would have been an obvious substitution for a base MIPI standard.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato for a transmitting device with circuit components to send image data of tiles comprising regions of interest utilizing an encoding scheme, such as MPEG-2 for packetizing data by further incorporating known elements of Fukuhara’s invention for transmitting image data, comprising image blocks for regions in an image, using an encoding scheme which transmits a payload comprising the region information because the combination of known elements for transmitting encoding data using known encoding schemes would result in predictable results with a high likelihood of success and because a person of ordinary skill in the art would reasonably infer that payload data is typically utilized for transmitting graphics and or the actual intended message such as image frame content. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato, Fukuhara, Raveendran, Nxp Semiconductors, Hwang, and Vembar by further incorporating known elements of Raveendran for a transmitting ROI packet data utilizing a MIPI transmitter because the prior art recognizes a mobile image device protocol for transmitting packet data comprising a packet structure utilizing Long Packets taught by MIPI in order to utilize a standardized protocol to efficiently image content as taught by Nxp Semiconductors.
Regarding claim 9, “a picture transmission device comprising: a detector that detects a region of overlap on a basis of information about respective ROIs (Regions Of Interest) in an image, the region of overlap in which two or more of the ROIs overlap each other; and a transmission circuitry that sends a plurality of pieces of third image data in Payload Data of a Long Packet and sends information about the respective ROIs in the image in Embedded Data, the plurality of pieces of third image data being obtained by omitting second image data of the region of overlap from a plurality of pieces of first image data of the ROIs in the image to avoid the second image data from being included redundantly in the plurality of pieces of first image data” is further rejected on obviousness grounds as discussed in the rejection of claims 1-2, 4-8 wherein the combination of Sato, Fukuhara, Raveendran, and Nxp Semiconductors discuss all the elements of claim 9 except the overlap feature of claim 9. For example, the rejection of claim 1 render obvious all the elements with respect to a transmission section that sends a plurality of pieces of third image data in Payload Data of a Long Packet and sends information about the respective ROIs in the image in Embedded Data. Furthermore, with respect to the overlapping elements of claim 9, Hwang further teaches “detects a region of overlap on a basis of information about respective ROIs (Regions Of Interest) in an image” and transmitting a plurality of pieces of third image data in Payload Data of a Long Packet and sends information about the respective ROIs in the image in Embedded Data, the plurality of pieces of third image data being obtained by omitting second image data of the region of overlap from a plurality of pieces of first image data of the ROIs in the image to avoid the second image data from being included redundantly in the plurality of pieces of first image data (see Hwang Fig. 16-17 and para 161-185 ROI Region overlaps the middle four regions of interest and the ROI info metadata for Residual data for ROI start point (X,Y) & (ROI_width X ROI height) and similarly in Fig. 17 and para 200-215 wherein ROI transmitted to the monitor comprises ROI which overlaps regions 3,4,5,6 renders obvious to discard/omit image data of the region of overlap from pieces of first image data of the ROIs in the image to avoid the redundant transmission of other regions in the plurality of pieces of first image data).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato for a transmitting device with circuit components to send image data of tiles comprising regions of interest utilizing an encoding scheme, for packetizing data by further incorporating known elements of Fukuhara’s invention for transmitting image data, comprising image blocks for regions in an image, using an encoding scheme which transmits a payload comprising the region information because the combination of known elements for transmitting encoding data using known encoding schemes would result in predictable results with a high likelihood of success and because a person of ordinary skill in the art would reasonably infer that payload data is typically utilized for transmitting graphics and or the actual intended message such as image frame content. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato and Fukuhara by further incorporating known elements of Raveendran for a mobile image device protocol for transmitting packets comprising regions of interest data because the prior art recognizes a mobile image device protocol for transmitting packet data comprising a packet structure utilizing Long Packets in order to utilize a standardized protocol to efficiently transmit regions of interest content as taught by Nxp Semiconductors. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato, Fukuhara, Raveendran, and Nxp Semiconductors because Vembar teaches prioritizing the transmission of regions of interest (ROIs) utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams related to the same content comprising a frame divided into viewports.
Regarding claim 10, “wherein the transmission circuitry sends the respective ROIs through virtual channels different from each other” Sato, Fukuhara, and Raveendran are silent with respect to virtual channels but Nxp Semiconductors recognizes that the purpose of a virtual channel identifier is to provide separate channels for different data flows that are interleaved in the data stream (Section 2.2.2.3) and Nxp Semiconductors Section 2.2.2 support for up to four interleaved virtual channels). The motivation of combine Sato, Fukuhara, Raveendran, and Nxp Semiconductors is further evidenced by Hwang (para 380-399 regions of interest transmitted utilizing different streams pertaining to ROIs for an image.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato for a transmitting device with circuit components to send image data of tiles comprising regions of interest utilizing an encoding scheme, such as MPEG-2 for packetizing data by further incorporating known elements of Fukuhara’s invention for transmitting image data, comprising image blocks for regions in an image, using an encoding scheme which transmits a payload comprising the region information because the combination of known elements for transmitting encoding data using known encoding schemes would result in predictable results with a high likelihood of success and because a person of ordinary skill in the art would reasonably infer that payload data is typically utilized for transmitting graphics and or the actual intended message such as image frame content. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato and Fukuhara by further incorporating known elements of Raveendran for a mobile image device protocol for transmitting packets comprising regions of interest data because the prior art recognizes a mobile image device protocol for transmitting packet data comprising a packet structure utilizing Long Packets in order to utilize a standardized protocol to efficiently transmit regions of interest content as taught by Nxp Semiconductors. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato, Fukuhara, Raveendran, and Nxp Semiconductors for utilizing a plurality of channels for transmitting regions of interest of an image in order to improve the transmission of high bandwidth content by utilizing more than one channel to transmit regions of interest that would require the transmission of higher definition data (e.g., overlapping regions) as taught by Hwang and because Vembar teaches prioritizing the transmission of regions of interest (ROIs) utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams related to the same content comprising a frame divided into viewports.
Regarding claim 12, “wherein the transmission circuitry puts data types of the respective ROIs in a packet header of the Payload Data, and sends the data type of the respective ROIs” is further rejected on obviousness grounds as discussed in the rejection of claim 1-2, 4-10 wherein Raveendran Fig. 7-9 and para 165-175. Whereas Raveendran does not specifically user the term “Long Packet” as claimed, the prior art recognizes the use of Long Packets when utilizing a mobile device to transmit packets data – see Nxp Semiconductors (Section 2.2.2 2.2.2.1 and teaches utilizing Payload Data of a Long Packet and header). Sato teaches header indicates whether stream comprises a ROI parameter data indicates a type (wherein the broadest reasonable interpretation of a type comprises the definition of “something distinguishable as a variety”).
Regarding claim 13, “wherein the transmission circuitry puts at least one of number of the ROIs included in the image, a region number of each of the ROIs, a data length of each of the ROIs, or an image format of each of the ROIs in the Payload Data, and sends the at least one of the number of the ROIs included in the image, the region number of each of the ROIs, the data length of each of the ROIs, or the image format of each of the ROIs” is further rejected on obviousness grounds as discussed in the rejection of claims 1-2, 4-10, 12 wherein Hwang teaches a region number in para 167.
Regarding claim 14, “wherein the transmission circuitry puts at least one of number of the ROIs included in the image, a region number of each of the ROIs, a data length of each of the ROIs, or an image format of each of the ROIs in a Data Field of a Short Packet, and sends the at least one of the number of the ROIs included in the image, the region number of each of the ROIs, the data length of each of the ROIs, or the image format of each of the ROIs” is further rejected on obviousness grounds as discussed in the rejection of claims 1-2, 4-10, 12-13 wherein Hwang teaches a region number in para 167.
Regarding claim 15, “wherein the transmission circuitry sends the image data of the ROIs in an image data frame and sends the information about the ROIs in a header or a footer of the image data frame” is further rejected on obviousness grounds as discussed in the rejection of claims 1-2, 4-10, 12-14 wherein Sato teaches image encoding device comprises a tile partitioning section for generating frames into memory (para 79, 90-93, 113, 122, 127-135, 146, 153, 156-160, 163, 169, 184-191 encoded stream of based layer and image of ROI tile is stored in the frame memory). See also Nxp Semiconductors Section 2.2.2 to 2.2.2.1 utilizing Payload Data of a Long Packet comprising a header and footer.
Regarding claim 16, “wherein the transmission circuitry sends a signal in MIPI (Mobile Industry Processor Interface) CSI (Camera Serial Interface)-2 specification, MIPI CSI-3 specification, or MIPI DSI (Display Serial Interface) specification” is further rejected on obviousness grounds as discussed in the rejection of claim 8 including claims 1-2, 4-10, 12-15.
Regarding claim 17, “a picture reception device comprising: a reception circuitry that receives a transmission signal including image data of ROIs (Regions Of Interest) in an image and information about the ROIs, the image data of the ROIs being included in Payload Data of a Long Packet, the information about the ROIs being included in Embedded Data; and an information processor that extracts information about the ROIs from the Embedded Data included in the transmission signal received by the reception circuitry, and extracts the image data of the ROIs from the Payload Data included in the transmission signal received by the reception circuitry on a basis of the extracted information; wherein the information about the ROIs indicates a region of overlap between two or more of the ROIs, and wherein the reception circuitry receives the ROIs through a mutually common virtual channel” the reception device of claim 17 is further rejected on obviousness grounds as discussed in the rejection of transmission device of claim 1 wherein the prior art renders obvious the elements of the transmission device and the prior art further teaches the transmission device communicates the claimed data to a reception device. For example, see Sato Fig. 23-26 and para 117-118, 123, 155, 162, 182, 188, 248, 253; Hwang Fig. 38-44 disclosing transmitter and receiver for implementing the invention of transmitting and receiving ROI in packetized data streams. With respect to circuitry, Sato para 252 references units understood as processing units of an encoding process wherein transmitters and receivers typically comprises hardware, software, circuitry or a combination of hardware, software, circuitry.
Regarding wherein the information about the ROIs indicates a region of overlap between two or more of the ROIs Sato, Fukuhara and Nxp Semiconductors do not disclose an overlapping feature as claimed. Hwang further teaches “ROIs indicates a region of overlap between two or more of the ROIs” and transmitting a plurality of pieces of third image data in Payload Data of a Long Packet and sends information about the respective ROIs in the image in Embedded Data, the plurality of pieces of third image data being obtained by omitting second image data of the region of overlap from a plurality of pieces of first image data of the ROIs in the image to avoid the second image data from being included redundantly in the plurality of pieces of first image data (see Hwang Fig. 16-17 and para 161-185 ROI Region overlaps the middle four regions of interest and the ROI info metadata for Residual data for ROI start point (X,Y) & (ROI_width X ROI height) and similarly in Fig. 17 and para 200-215 wherein ROI transmitted to the monitor comprises ROI which overlaps regions 3,4,5,6 renders obvious to discard/omit image data of the region of overlap from pieces of first image data of the ROIs in the image to avoid the redundant transmission of other regions in the plurality of pieces of first image data). In an analogous art, Thomas teaches identifying multiple regions of interest with overlapping content (para 44-48) and including the data in payload content (para 59, 117, 121, 134).
With respect to an image divided into multiple regions of interest, Hwang teaches the divided images are transmitted through respective streams (para 12, 83, 131, 132, 175, 178, 217, 225, 310, 381) and also teaches that a plurality of virtual channels are transmitted as part of a transport stream (see para 301, 315-317, 328-343. See also Hwang para 381 - each elementary stream with respect to each image included in the service may include stream_info_details( ), and the stream_info_details( ) may signal information about the streams with respect to the panoramic image. In the panorama service according to an embodiment of the present invention, information about each image may be signaled using the stream_info_details( ) according to an embodiment of the present invention even when the stream_type is MPEG2, AVC, HEVC or the like and Hwang para 386 - component_count field may indicate the number of components (streams) included in a relevant channel (service). Whereas Hwang as discussed supra discloses that multiple streams are multiplexed as a number of components (streams) included in a relevant channel (service), as discussed above, Nxp Semiconductors recognizes that the purpose of a virtual channel identifier is to provide separate channels for different data flows that are interleaved in the data stream (Section 2.2.2.3).
In an analogous art, Vembar teaches prioritizing the transmission of regions of interest (ROIs) utilizing parallel processing wherein para 55-57 - utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams; see also para 166 dynamically resize and assign priorities to viewports and transmit rendered viewports as they are ready (e.g. while other viewports render). For example, the PPU 713 may include similar components and/or features as the parallel processor 200, further configured with a graphics viewport apparatus as described herein. The system 700 may also be adapted to work with a stereo head mounted system such as, for example, the system described in connection with FIGS. 11-15 below. In particular, the HMII 1100 described in more detail below may include a gaze tracker to provide gaze/focus information to the frame divider 721 and/or viewport prioritizer 722. See also para 170 a prioritized region of interest (ROI) for wireless VR use cases and applications, so there is a priority associated with which region of the image is being encoded and decoded. In some embodiments, eye tracking information may be utilized to identify a prioritized ROI. Some embodiments may provide similar benefits for a wired VR system (e.g. where transmission occurs over a cable). See also para 180 some embodiments may provide individual rendering frequencies for a wireless VR. Objects in the periphery may not need to be rendered at the same rate as the center of the screen. Some embodiments may render different blocks of an image at different fps for prioritized rendering and display. Advantageously, some embodiments may reduce the amount of work being done or allow doing more work at the center or ROI of the image. Some embodiments may blend content at the edges of the different viewports (e.g. blending new data with stale pixel data). For example, a HMD may include sensors to provide eye/gaze tracking information which the GPU/CPU may use to change the ROI dynamically and adjust the viewports.
See prior art made of record but not relied upon to avoid duplicative references: Thomas et al., PG Pub 2018/0295400 (hereafter Thomas) teaches identifying multiple regions of interest with overlapping content (para 44-48) and including the data in payload content (para 59, 117, 121, 134).
Regarding claim 18, “wherein the information processor detects the region of overlap in which the two or more of the ROIs overlap each other on a basis of the extracted information, and extracts image data of the respective ROIs from the Payload Data included in the transmission signal received by the reception circuitry on a basis of the extracted information and information of the detected region of overlap” the claim is rejected on obviousness grounds as discussed in the rejection of claims 1-2, 4-10, 12-17. Wherein the rejection of claim 17 does not discuss the element with respect to the “overlap”, in haec verba, (i.e., detects a region of overlap in which two or more of the ROIs overlap each other on a basis of the extracted information, and extracts image data of the respective ROIs from the Payload Data included in the transmission signal received by the reception circuitry on a basis of the extracted information and information of the detected region of overlap).
For example, the rejection of claims 1 and 17 render obvious all the elements with respect to a transmission circuitry that sends a plurality of pieces of third image data in Payload Data of a Long Packet and sends information about the respective ROIs in the image in Embedded Data. Furthermore, with respect to the overlapping elements of claim 9, Hwang further teaches “detects a region of overlap on a basis of information about respective ROIs (Regions Of Interest) in an image” and transmitting a plurality of pieces of third image data in Payload Data of a Long Packet and sends information about the respective ROIs in the image in Embedded Data, the plurality of pieces of third image data being obtained by omitting second image data of the region of overlap from a plurality of pieces of first image data of the ROIs in the image to avoid the second image data from being included redundantly in the plurality of pieces of first image data (see Hwang Fig. 16-17 and para 161-185 ROI Region overlaps the middle four regions of interest and the ROI info metadata for Residual data for ROI start point (X,Y) & (ROI_width X ROI height) and similarly in Fig. 17 and para 200-215 wherein ROI transmitted to the monitor comprises ROI which overlaps regions 3,4,5,6 renders obvious to discard/omit image data of the region of overlap from pieces of first image data of the ROIs in the image to avoid the redundant transmission of other regions in the plurality of pieces of first image data).
In an analogous art, Vembar teaches prioritizing the transmission of regions of interest (ROIs) utilizing parallel processing wherein para 55-57 - utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams; see also para 166 dynamically resize and assign priorities to viewports and transmit rendered viewports as they are ready (e.g. while other viewports render). For example, the PPU 713 may include similar components and/or features as the parallel processor 200, further configured with a graphics viewport apparatus as described herein. The system 700 may also be adapted to work with a stereo head mounted system such as, for example, the system described in connection with FIGS. 11-15 below. In particular, the HMII 1100 described in more detail below may include a gaze tracker to provide gaze/focus information to the frame divider 721 and/or viewport prioritizer 722. See also para 170 a prioritized region of interest (ROI) for wireless VR use cases and applications, so there is a priority associated with which region of the image is being encoded and decoded. In some embodiments, eye tracking information may be utilized to identify a prioritized ROI. Some embodiments may provide similar benefits for a wired VR system (e.g. where transmission occurs over a cable). See also para 180 some embodiments may provide individual rendering frequencies for a wireless VR. Objects in the periphery may not need to be rendered at the same rate as the center of the screen. Some embodiments may render different blocks of an image at different fps for prioritized rendering and display. Advantageously, some embodiments may reduce the amount of work being done or allow doing more work at the center or ROI of the image. Some embodiments may blend content at the edges of the different viewports (e.g. blending new data with stale pixel data). For example, a HMD may include sensors to provide eye/gaze tracking information which the GPU/CPU may use to change the ROI dynamically and adjust the viewports.
See prior art made of record but not relied upon to avoid duplicative references: Thomas et al., PG Pub 2018/0295400 (hereafter Thomas) teaches identifying multiple regions of interest with overlapping content (para 44-48) and including the data in payload content (para 59, 117, 121, 134).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato for a transmitting device with circuit components to send image data of tiles comprising regions of interest utilizing an encoding scheme, for packetizing data by further incorporating known elements of Fukuhara’s invention for transmitting image data, comprising image blocks for regions in an image, using an encoding scheme which transmits a payload comprising the region information because the combination of known elements for transmitting encoding data using known encoding schemes would result in predictable results with a high likelihood of success and because a person of ordinary skill in the art would reasonably infer that payload data is typically utilized for transmitting graphics and or the actual intended message such as image frame content. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato and Fukuhara by further incorporating known elements of Raveendran for a mobile image device protocol for transmitting packets comprising regions of interest data because the prior art recognizes a mobile image device protocol for transmitting packet data comprising a packet structure utilizing Long Packets in order to utilize a standardized protocol to efficiently transmit regions of interest content as taught by Nxp Semiconductors. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato, Fukuhara, Raveendran, and Nxp Semiconductors by further incorporating known elements of Hwang for utilizing more than one channel to transmit regions of interest that would require the transmission of higher definition data as suggested by Hwang when transmitting extracted overlapping regions of interest and provided in a higher definition format than the image with the plurality ROI because Vembar teaches prioritizing the transmission of regions of interest (ROIs) utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams related to the same content comprising a frame divided into viewports.
Regarding claim 19, “wherein the transmission circuitry receives a signal in MIPI (Mobile Industry Processor Interface) CSI (Camera Serial Interface)-2 specification, MIPI CSI-3 specification, or MIPI DSI (Display Serial Interface) specification” Sato and Fukuhara are silent with respect to MIP specifications. However, in an analogous art, Raveendran teaches a transmitting ROI utilizing a MIPI transmitter (Fig. 7-9 and para 53, 165-175). Even where Raveendran does not specifically CSI-2 or CSI-3, a person of ordinary skill would reasonably infer that a MIPI transceiver utilizing any amended MIPI standard would have been an obvious substitution for a base MIPI standard.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato for a transmitting device with circuit components to send image data of tiles comprising regions of interest utilizing an encoding scheme, such as MPEG-2 for packetizing data by further incorporating known elements of Fukuhara’s invention for transmitting image data, comprising image blocks for regions in an image, using an encoding scheme which transmits a payload comprising the region information because the combination of known elements for transmitting encoding data using known encoding schemes would result in predictable results with a high likelihood of success and because a person of ordinary skill in the art would reasonably infer that payload data is typically utilized for transmitting graphics and or the actual intended message such as image frame content. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sato, Fukuhara, Hwang, and Vembar by further incorporating known elements of Raveendran for a transmitting ROI packet data utilizing a MIPI transmitter because the prior art recognizes a mobile image device protocol for transmitting packet data comprising a packet structure utilizing Long Packets taught by MIPI in order to utilize a standardized protocol to efficiently image content as taught by Nxp Semiconductors.
Regarding claim 20, wherein the Embedded Data is separate and distinct from a packet header of the Payload Data, is further rejected on obviousness grounds as discussed in the rejection of claims 1, 7-8, 17, 19 wherein Sato teaches image encoding device comprises a tile partitioning circuitry for generating frames into memory (para 79, 90-93, 113, 122, 127-135, 146, 153, 156-160, 163, 169, 184-191 encoded stream of based layer and image of ROI tile is stored in the frame memory). See also Nxp Semiconductors Section 2.2.2 to 2.2.2.1 utilizing Payload Data of a Long Packet comprising a header and footer. A person of ordinary skill in the art would have understood that embedded data of a Payload data may be transmitted as a footer which is different from a header.
Regarding claim 21, “wherein each ROI (Region of Interest) of the ROIs is a selected image region of the image, and the selected image region of the image is less than all of the image” Hwang teaches that the panoramic image (main image) may be divided into left, center and right regions. As shown in the figure, the panoramic image may be divided into a plurality of images in a mosaic form (Hwang para 115) and further teaches indicating the number of Regions of Interest being transmitted (para 180-183, 222). More importantly, Hwang discloses elements with respect to enhancement signals of the ROI is merged with a base layer (para 83, 92-96, 111-112, 162-166). As such, the enhanced region of interest is composed of sections of different regions of the display image that are of interest to the user. For example, the enhanced region of interest is a portion of each of the tiles that are of interest to the receiving user. See also Vembar teaches prioritizing the transmission of regions of interest (ROIs) utilizing parallel processing wherein para 55-57 - utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams; see also para 166 dynamically resize and assign priorities to viewports and transmit rendered viewports as they are ready (e.g. while other viewports render). For example, the PPU 713 may include similar components and/or features as the parallel processor 200, further configured with a graphics viewport apparatus as described herein. The system 700 may also be adapted to work with a stereo head mounted system such as, for example, the system described in connection with FIGS. 11-15 below. In particular, the HMII 1100 described in more detail below may include a gaze tracker to provide gaze/focus information to the frame divider 721 and/or viewport prioritizer 722. See also para 170 a prioritized region of interest (ROI) for wireless VR use cases and applications, so there is a priority associated with which region of the image is being encoded and decoded. In some embodiments, eye tracking information may be utilized to identify a prioritized ROI. Some embodiments may provide similar benefits for a wired VR system (e.g. where transmission occurs over a cable). See also para 180 some embodiments may provide individual rendering frequencies for a wireless VR. Objects in the periphery may not need to be rendered at the same rate as the center of the screen. Some embodiments may render different blocks of an image at different fps for prioritized rendering and display. Advantageously, some embodiments may reduce the amount of work being done or allow doing more work at the center or ROI of the image. Some embodiments may blend content at the edges of the different viewports (e.g. blending new data with stale pixel data). For example, a HMD may include sensors to provide eye/gaze tracking information which the GPU/CPU may use to change the ROI dynamically and adjust the viewports.
See prior art made of record but not relied upon to avoid duplicative references: Thomas et al., PG Pub 2018/0295400 (hereafter Thomas) teaches identifying multiple regions of interest with overlapping content (para 44-48) and including the data in payload content (para 59, 117, 121, 134).
Regarding claim 22, “wherein the ROIs include a first ROI and one or more additional ROIs, and the image data includes a first portion of the image data corresponding the first ROI, and a second portion of the image data corresponding to less than all of the one or more additional ROIs, wherein the information about the ROIs indicates a region of overlap between the first ROI and the one or more additional ROIs, wherein the first portion of the image data includes image data of the region of overlap, wherein the second portion of the image data does not include the image data of the region of overlap, and wherein the first portion and the second portion of the image data are each less than an entirety of the image” is further rejected on obviousness grounds as discussed in the rejection of claims 1-2, 4-10, 12-21. The limitations of claim 22 incorporate limitations taken from claims 1-2, 4-10, 12-21 and are rejected as discussed in the obviousness rejection of claims 1-2, 4-10, 12-21. Based on the interpretation of the limitations of claim 22 (i.e., “wherein the second portion of the image data does not include the image data of the region of overlap), there is not an explicit recited limitation in claim 22 that indicates that the each/every one of the “one or more additional ROIs” that is/are associated with the claimed “second portion” would overlap with the first portion of the image data corresponding to the first ROI. Stated differently, a person of ordinary skill would reasonably infer that not every permutation of each and every displayed ROI would necessarily overlap because the applicant has claimed a plurality of ROIs (i.e., a first ROI and one or more additional ROIs) and would only include overlap information in the event that certain ROIs happen to overlap. Therefore, a person of ordinary skill in the art would reasonably infer that a ROI that does not overlap with any other ROI would not comprise overlap information.
As discussed in claim 21, and wherein claim 21 is relevant to the analysis of the limitations of claim 22 (i.e., “wherein each ROI (Region of Interest) of the ROIs is a selected image region of the image, and the selected image region of the image is less than all of the image”), Hwang teaches that the panoramic image (main image) may be divided into left, center and right regions. As shown in the figure, the panoramic image may be divided into a plurality of images in a mosaic form (Hwang para 115) and further teaches indicating the number of Regions of Interest being transmitted (para 180-183, 222). More importantly, Hwang discloses elements with respect to enhancement signals of the ROI is merged with a base layer (para 83, 92-96, 111-112, 162-166). As such, the enhanced region of interest is composed of sections of different regions of the display image that are of interest to the user. For example, the enhanced region of interest is a portion of each of the tiles that are of interest to the receiving user. See also Vembar teaches prioritizing the transmission of regions of interest (ROIs) utilizing parallel processing wherein para 55-57 - utilizing parallel processing to efficiency use the available bandwidth comprising utilizing virtual channels to separate traffic streams; see also para 166 dynamically resize and assign priorities to viewports and transmit rendered viewports as they are ready (e.g. while other viewports render). For example, the PPU 713 may include similar components and/or features as the parallel processor 200, further configured with a graphics viewport apparatus as described herein. The system 700 may also be adapted to work with a stereo head mounted system such as, for example, the system described in connection with FIGS. 11-15 below. In particular, the HMII 1100 described in more detail below may include a gaze tracker to provide gaze/focus information to the frame divider 721 and/or viewport prioritizer 722. See also para 170 a prioritized region of interest (ROI) for wireless VR use cases and applications, so there is a priority associated with which region of the image is being encoded and decoded. In some embodiments, eye tracking information may be utilized to identify a prioritized ROI. Some embodiments may provide similar benefits for a wired VR system (e.g. where transmission occurs over a cable). See also para 180 some embodiments may provide individual rendering frequencies for a wireless VR. Objects in the periphery may not need to be rendered at the same rate as the center of the screen. Some embodiments may render different blocks of an image at different fps for prioritized rendering and display. Advantageously, some embodiments may reduce the amount of work being done or allow doing more work at the center or ROI of the image. Some embodiments may blend content at the edges of the different viewports (e.g. blending new data with stale pixel data). For example, a HMD may include sensors to provide eye/gaze tracking information which the GPU/CPU may use to change the ROI dynamically and adjust the viewports.
See prior art made of record but not relied upon to avoid duplicative references: Thomas et al., PG Pub 2018/0295400 (hereafter Thomas) teaches identifying multiple regions of interest with overlapping content (para 44-48) and including the data in payload content (para 59, 117, 121, 134).
Conclusion
Prior art made of record but not relied upon: Sadi et al., US11205305B2
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALFONSO CASTRO whose telephone number is (571)270-3950. The examiner can normally be reached on Monday to Friday from 10am to 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nathan Flynn can be reached. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALFONSO CASTRO/Primary Examiner, Art Unit 2421